Tool calling (Function / Tool Calling)
π Short definition
A mechanism where the LLM chooses and parameterizes an external function (JSON schema) instead of answering directly: search, calculation, database query, script execution1.
π Detailed definition
Tool calling (function calling / tool calling) turns the LLM into an orchestrator: the model receives a list of described tools (name, parameters, types), decides which to invoke, produces a structured payload, then integrates the result into the rest of its reasoning.
In an on-premise stack, the engine (vLLM, Ollama, SGLang) often exposes an OpenAI-compatible βtoolsβ API; the application actually executes the tool server-side (never with blind trust).
Typical pattern: a fast small model for tool routing, a large model for the final synthesis β see π§© RAG & Agents.
π‘ Why it matters for on-prem AI
- Foundation of autonomous agents and agentic RAG (retrieve, filter, rewrite).
- Connects the LLM to internal systems without fine-tuning: tool contracts = security boundary.
- Related to OWASP LLM06 (Excessive Agency): attack surface if too many tools or excessive permissions.
β οΈ Common pitfalls
- Giving the agent destructive tools (shell, DELETE SQL) without sandbox or HITL validation.
- Parsing βalmost validβ JSON without a downstream validator β prefer SGLang or strict constraints for structured outputs.
- Ignoring latency: each tool turn = new prefill; optimize shared context (RadixAttention).
π To go deeper
- π§© RAG & Agents
- π Inference security β LLM06 Excessive Agency
- LangGraph β multi-step orchestration
π See also
Footnotes
-
OpenAI API β Function calling (de facto pattern adopted by vLLM/Ollama). https://developers.openai.com/api/docs/guides/function-calling β©