Function calling, also called tool use, is the API mechanism where the LLM decides — based on the conversation — to call one of your code functions with structured arguments instead of generating free text. The model emits a structured "I want to call function X with arguments " message; your code runs the function; the result goes back to the model; the model continues. It is the foundation of every AI agent built in 2026.
The basic loop:
- You define a set of available functions with names, descriptions and JSON schemas for arguments.
- You include those definitions in the API call alongside the user message.
- The model returns either a text response or a structured "call function X" message.
- Your code executes the call (database query, API request, calculation, file read, etc.).
- You send the result back to the model.
- The model continues, potentially calling more functions or finally responding to the user.
Why function calling matters:
- Determinism for the parts that should be deterministic — math, dates, exact data lookups go to code, not to the model's pattern matching.
- Real-time data — the model can fetch fresh information instead of relying on its training cutoff.
- Action in the world — booking calendars, sending emails, posting to APIs, controlling browsers all become possible.
- Reduced hallucination — the model does not have to guess about facts it can look up.
The 2026 landscape:
- OpenAI tool calling — supports parallel calls (multiple functions in one turn), forced calls (you require a specific function), and strict mode (guaranteed schema-valid arguments).
- Anthropic tool use — similar capabilities, including fine-grained control over when the model is allowed to respond directly vs must call a tool.
- Google function calling — supported across Vertex and the Gemini app.
- Open-weight models — most modern instruct models (Llama 3.1+, Mistral Large, DeepSeek V3) support function calling natively; older ones need parser glue.
- Model Context Protocol (MCP) — Anthropic-led open standard for connecting tools to any compatible LLM.
The practical patterns:
- Read tools for retrieval (search, query database, fetch URL).
- Write tools for action (send email, create calendar event, update record).
- Compute tools for determinism (math, currency conversion, date arithmetic).
- Meta tools for orchestration (delegate to another agent, list available capabilities).
For a US engineering team in 2026, function calling has become the default architecture for any AI feature that needs to do more than chat. The shift from "LLM generates text" to "LLM orchestrates tools" is the technical foundation of the entire agent wave, and the providers' tool-calling APIs are mature enough that you rarely need a heavy framework on top.
