Skip to content

Tool calling (Function / Tool Calling)

πŸ“ Short definition

A mechanism where the LLM chooses and parameterizes an external function (JSON schema) instead of answering directly: search, calculation, database query, script execution1.

πŸ“– Detailed definition

Tool calling (function calling / tool calling) turns the LLM into an orchestrator: the model receives a list of described tools (name, parameters, types), decides which to invoke, produces a structured payload, then integrates the result into the rest of its reasoning.

In an on-premise stack, the engine (vLLM, Ollama, SGLang) often exposes an OpenAI-compatible β€œtools” API; the application actually executes the tool server-side (never with blind trust).

Typical pattern: a fast small model for tool routing, a large model for the final synthesis β€” see 🧩 RAG & Agents.

πŸ’‘ Why it matters for on-prem AI

  • Foundation of autonomous agents and agentic RAG (retrieve, filter, rewrite).
  • Connects the LLM to internal systems without fine-tuning: tool contracts = security boundary.
  • Related to OWASP LLM06 (Excessive Agency): attack surface if too many tools or excessive permissions.

⚠️ Common pitfalls

  • Giving the agent destructive tools (shell, DELETE SQL) without sandbox or HITL validation.
  • Parsing β€œalmost valid” JSON without a downstream validator β€” prefer SGLang or strict constraints for structured outputs.
  • Ignoring latency: each tool turn = new prefill; optimize shared context (RadixAttention).

πŸ“š To go deeper

  1. 🧩 RAG & Agents
  2. πŸ” Inference security β€” LLM06 Excessive Agency
  3. LangGraph β€” multi-step orchestration

πŸ”— See also

Footnotes

  1. OpenAI API β€” Function calling (de facto pattern adopted by vLLM/Ollama). https://developers.openai.com/api/docs/guides/function-calling ↩

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.