Function Calling
Master function calling in LLMs: tool use, API integration, and building AI agents that can interact with external systems.
What is function calling?
Function calling is a pattern in which a language model proposes a named tool and structured arguments, then application code decides whether and how to execute that proposal. The tool might read an order, calculate a price, reserve inventory, or send a message.
The model does not call your database or API by itself. It produces data shaped like {"name":"lookup_order","arguments":{"orderId":"ord_1842"}}. Your application validates that data, checks the caller's authority, executes trusted code, and returns a structured result for the model to explain.
Function calling matters because generated text can now influence real systems. A fluent answer can be merely wrong; a poorly controlled tool call can expose another tenant's data, repeat a charge, or delete a record.
Core invariant: the model may propose an action, but deterministic application code owns validation, authorization, execution, and side effects.
Review Conversational AI first if conversation history and model turns are unfamiliar.
One model turn becomes a controlled execution loop
A tool-enabled response is an intermediate protocol message, not the final answer and not permission to act.
The application stays in control
Each boundary turns probabilistic model output into a typed, authorized, observable operation.
Intent
User request
The application attaches identity, tenant, conversation state, and only the tools allowed for this task.
Model output
Tool proposal
The model chooses a tool and emits arguments that claim to match its input schema.
Trust boundary
Deterministic gate
Code validates the schema, permission, resource scope, risk class, confirmation, and budget.
Capability
Tool execution
Trusted code runs with scoped credentials, a deadline, idempotency controls, and an audit identifier.
Result
Typed observation
The application returns bounded data or a structured error. The model can explain, ask a question, or propose another call.
Three values must remain separate
- Proposal: what the model requested, preserved exactly for inspection.
- Decision: why deterministic policy allowed, denied, or paused the request.
- Observation: what the tool actually returned or changed in the authoritative system.
Never replace the observation with the model's assumption that a call probably succeeded.
A tool contract has two boundaries
The model-facing contract helps the model choose and populate a tool. The execution policy decides whether the proposed capability is available to this caller in this context. JSON Schema can constrain shape; it cannot prove business authority.
Selection
Name and description
State what the tool does, when to use it, when not to use it, and what its result means. Ambiguous tools create routing errors before validation starts.
Shape
Input schema
Declare types, required fields, enums, ranges, formats, and whether unknown properties are rejected. Keep arguments smaller than the underlying API.
Authority
Execution policy
Bind the call to identity, tenant, task, permission, risk, budget, and confirmation. These checks use trusted application context, not model-supplied claims.
Observation
Result contract
Return bounded fields, stable status codes, provenance, and side-effect identifiers. Do not dump raw internal records back into the context window.
According to the JSON Schema object reference, properties are not required unless named in required, and extra properties are allowed unless constrained. Provider APIs use different envelopes, but both OpenAI function tools and Anthropic client tools describe tool inputs with JSON Schema.
Trace a proposal through schema and policy
Loading the contract scenarios...
Loading contract scenarios...
Design the narrowest useful schema
A good schema reduces ambiguity without pretending to replace runtime policy. Prefer one operation per tool and make invalid states difficult to express.
Put these constraints in the schema
- Require identifiers and values that execution cannot infer safely.
- Use enums for small closed choices such as
summaryversusshipping_status. - Bound string length, collection size, numeric range, and nesting depth.
- Reject unknown fields when the provider's supported JSON Schema subset allows it.
- Describe units, identifier formats, and omitted-value behavior in plain language.
Keep these checks in trusted policy code
- Whether the caller may use the tool at all.
- Whether an order, account, or file belongs to the active tenant.
- Whether a write requires approval, separation of duties, or a fresh login.
- Which credential, region, network path, rate limit, and cost budget apply.
- Whether the operation may be retried and how duplicate effects are prevented.
Schema conformance makes an argument structurally acceptable. It does not make the request true, authorized, safe, or current.
The runnable example rejects unknown arguments at the schema boundary, then applies tenant and permission checks separately.
Execute proposals as a state machine
Tool orchestration becomes reliable when every call moves through explicit states and produces durable evidence.
1 Proposed
Parse and correlate
Capture the provider's call ID, tool name, raw arguments, conversation turn, and task identity. Reject malformed or unknown calls.
2 Decided
Validate and authorize
Apply schema, identity, tenant, permission, approval, budget, and risk checks before obtaining an execution credential.
3 Attempted
Execute with a deadline
Pass only validated arguments, use least-privilege credentials, and attach an idempotency key to supported writes.
4 Recorded
Observe and continue
Store status, latency, side-effect ID, and bounded result. Then return the matching tool result or structured error to the model.
The call identifier is a correlation key
When a model emits several tool calls, each result must be matched to the correct proposal. Preserve call IDs across queues, workers, logs, and the next model turn. Do not rely on array position after parallel execution.
Recover a timeout without repeating the user's intent
Loading the execution scenarios...
Treat timeouts as unknown outcomes
A timeout tells the caller that no response arrived before its deadline. It does not prove that the downstream operation failed. The service may have committed a write and lost the response.
Safe to repeat
Read-only call
A repeated lookup creates no external mutation, though it still consumes capacity and may observe a newer state.
Repeat one intent
Idempotent write
The caller sends a stable idempotency key. The service stores the first outcome and returns it for duplicate attempts instead of applying the effect again.
Reconcile first
Non-idempotent write
After an ambiguous timeout, query authoritative state or route to review. Blind retry can create a second message, booking, or payment.
The next example simulates a response lost after commit. Run it with Node to see two delivery attempts resolve to one stored side effect.
Parallel and sequential calls express different dependencies
Use parallel calls only when their inputs and side effects are independent. If one result determines another call's arguments or permission, keep the calls sequential.
Choose execution order from data dependencies
Parallelism reduces latency only when calls do not depend on one another and partial failure has a defined policy.
Parallel
Independent reads
Fetch weather and exchange rates together, correlate both results, and tolerate one bounded partial failure if the product can explain it.
Sequential
Dependent lookup
Resolve an account before requesting its invoices because the first observation supplies the authorized account ID.
Ordered
Consequential writes
Confirm intent, reserve resources, commit the write, and verify the final state. Do not fan out writes without compensation or idempotency.
Define partial-failure behavior before enabling concurrency
- Decide whether one failed read cancels, degrades, or leaves the other results usable.
- Bound concurrency so one model turn cannot exhaust a dependency or cost budget.
- Preserve one result record per call ID, including denied and timed-out calls.
- Return concise errors the model can act on without exposing secrets or internal stack traces.
Test decisions, not just happy-path handlers
Unit tests for the underlying API wrapper are necessary but insufficient. The model and control plane must be evaluated together.
Contract and routing tests
- Valid, missing, mistyped, oversized, enum-invalid, and unknown arguments.
- Similar tool names, ambiguous requests, missing user information, and requests that need no tool.
- Provider model or schema-version changes against a frozen routing evaluation set.
Authority and side-effect tests
- Cross-tenant identifiers, revoked permissions, expired approvals, and model-supplied privilege fields.
- Timeout before commit, timeout after commit, duplicate delivery, worker restart, and result replay.
- Parallel partial failure, cancellation, out-of-order completion, and call-ID correlation.
Production signals
Selection rate
Tool chosen by task slice
Denial rate
Schema and policy separately
p95 latency
Model, queue, and tool stages
Duplicate effects
Target exactly zero
Record tool name and schema version, call and task IDs, policy decision, attempt count, latency, status code, idempotency key hash, side-effect ID, and bounded result metadata. Redact secrets and minimize sensitive arguments.
Production readiness checklist
- Expose only the smallest tool set needed for the current task and caller.
- Version tool descriptions, input schemas, result schemas, and policy together.
- Keep authorization context and credentials outside model-controlled arguments.
- Require confirmation for consequential, irreversible, exceptional, or high-value actions.
- Set per-call deadlines, per-task budgets, concurrency limits, and explicit stop conditions.
- Make write retries idempotent or reconcile authoritative state before another attempt.
- Treat tool results as untrusted input: validate, bound, redact, and preserve provenance.
- Audit every proposal, denial, attempt, result, and side effect with stable correlation IDs.
- Provide a kill switch and a deterministic fallback for unavailable or degraded tools.