How Tool Calling Works in an AI Agent Loop

By Numan ·

Follow the model request, application execution, tool result and final answer in a basic agent loop.

How Tool Calling Works in an AI Agent Loop

Tool calling can seem magical until you trace the messages. The OpenAI function-calling guide (https://developers.openai.com/api/docs/guides/function-calling) describes a simple sequence: the application offers tools, the model requests one, application code executes it, the result goes back to the model, and the model continues or answers.

Walk through one request

Suppose a user asks for the latest issue in a repository. The model may request list_issues(repo, limit=1). Your application checks that the repository is allowed, calls the service, and returns a compact result. The model then writes an answer grounded in that result. The model did not directly open a database connection; your application performed the operation.

Where to put the controls

At the tool boundary, validate input, authenticate the caller, authorize the target, and limit result size. On every call, record the tool name, target, and outcome. The model may request another tool after seeing the first result, so the budget should cover a sequence, not only a single call.

Keep the result useful

Return fields the model needs, plus a stable source reference where possible. Avoid sending raw secrets or a huge document when a small excerpt will answer the question. If the backend fails, return a clear error instead of a fabricated empty list.

One useful test

Ask the agent to find a record it cannot access. The expected behavior is a permission denial that the final answer reflects accurately. If the model answers as though it saw the record, the application should treat that as an evaluation failure.

MCP can expose tools to multiple hosts, but the call loop and service-side responsibility remain. See MCP vs function calling (https://ubits.tech/learning/p/mcp-vs-function-calling-ai-agents).

Primary source: OpenAI function calling (https://developers.openai.com/api/docs/guides/function-calling).