Function calling — giving LLMs hands — Agentic AI Engineering

Why this matters

Function calling — also called tool use — is the single most important capability in agentic AI. It is how an LLM goes from "answering questions" to "doing things." Every agent loop you build for the rest of your career runs on top of function calling.

This lesson teaches the raw mechanics. Frameworks (LangGraph, CrewAI, OpenAI Agents SDK) are sugar on top of what you learn here.

Learning objectives

Define tools with clean Pydantic schemas and dispatch them yourself.
Build a manual ReAct loop in 50 lines with no framework.
Use OpenAI's automatic tool-runner and Anthropic's tool_runner helper.
Handle parallel tool calls, tool errors, and infinite-loop guards.

1. The mental model

The LLM does not call your code. It returns a JSON saying "I want to call function X with arguments Y." Your code runs the function, then sends the result back to the LLM in a new turn. The LLM either calls another tool or returns a final answer.

[user message]
    ↓
LLM → tool_call(X, Y)
    ↓
your code runs X(Y) → result
    ↓
[new turn with tool_result]
    ↓
LLM → final answer  OR  tool_call(another)

That cycle, repeated, is the agent loop.

2. Defining a tool

A tool needs:

A unique name (snake_case verb + noun).
A clear description (one sentence, what it does).
A JSON Schema for the arguments (Pydantic gives you this).
A Python implementation.

Pydantic-first pattern:

python

from pydantic import BaseModel, Field

class GetWeatherArgs(BaseModel):
    city: str = Field(..., description="City name like 'Mumbai' or 'Tokyo'")
    unit: str = Field("celsius", description="celsius or fahrenheit")

def get_weather(args: GetWeatherArgs) -> dict:
    # imagine a real API
    return {"city": args.city, "temp": 32, "unit": args.unit}

The schema you give to the LLM:

python

TOOL_SCHEMA = {
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": GetWeatherArgs.model_json_schema(),
        "strict": True,  # OpenAI strict mode
    }
}

For Anthropic the shape is {"name": ..., "description": ..., "input_schema": ...}.

3. Manual loop with OpenAI Chat Completions

50 lines, no framework:

python

import json
from openai import OpenAI

client = OpenAI()
TOOL_REGISTRY = {"get_weather": get_weather}

def run_agent(user_msg: str, max_steps: int = 5):
    history = [
        {"role": "system", "content": "You are a helpful assistant. Use tools when needed."},
        {"role": "user", "content": user_msg},
    ]
    for step in range(max_steps):
        resp = client.chat.completions.create(
            model="gpt-4.1-mini",
            messages=history,
            tools=[TOOL_SCHEMA],
            tool_choice="auto",
        )
        msg = resp.choices[0].message
        history.append(msg.model_dump(exclude_unset=True))

        if not msg.tool_calls:
            return msg.content  # final answer

        for call in msg.tool_calls:
            args = GetWeatherArgs(**json.loads(call.function.arguments))
            result = TOOL_REGISTRY[call.function.name](args)
            history.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": json.dumps(result),
            })

    return "ran out of steps"

print(run_agent("What is the weather in Mumbai?"))

This is the entire idea. Frameworks add: state persistence, multi-agent, evals, observability.

4. Manual loop with OpenAI Responses API

Cleaner because each output item has a type:

python

TOOL = {
    "type": "function",
    "name": "get_weather",
    "description": "Get the current weather for a city.",
    "parameters": GetWeatherArgs.model_json_schema(),
    "strict": True,
}

def run_responses(user_msg: str, max_steps: int = 5):
    input_items = [{"role": "user", "content": user_msg}]
    for _ in range(max_steps):
        r = client.responses.create(
            model="gpt-4.1-mini",
            instructions="You are a helpful assistant.",
            input=input_items,
            tools=[TOOL],
        )
        input_items.extend(r.output)
        calls = [i for i in r.output if i.type == "function_call"]
        if not calls:
            return r.output_text
        for call in calls:
            args = GetWeatherArgs(**json.loads(call.arguments))
            result = get_weather(args)
            input_items.append({
                "type": "function_call_output",
                "call_id": call.call_id,
                "output": json.dumps(result),
            })

5. Anthropic's `tool_runner` (the easy mode)

Anthropic ships a Python helper that runs the loop for you:

python

import anthropic

aclient = anthropic.Anthropic()

@anthropic.beta_tool
def get_weather(city: str, unit: str = "celsius") -> dict:
    """Get the current weather for a city."""
    return {"city": city, "temp": 32, "unit": unit}

runner = aclient.beta.messages.tool_runner(
    max_tokens=1024,
    model="claude-sonnet-4-6",
    tools=[get_weather],
    messages=[{"role": "user", "content": "Weather in Mumbai?"}],
)

for msg in runner:
    print(msg)
print(runner.until_done().content[0].text)

The decorator builds the schema from the function's signature and docstring. The runner iterates until Claude is done.

6. Parallel tool calls

Modern LLMs can request multiple tool calls in a single turn. Run them in parallel for speed:

python

import asyncio

async def run_calls_parallel(calls):
    async def run_one(call):
        args = TOOL_ARGS_BY_NAME[call.function.name](**json.loads(call.function.arguments))
        fn = TOOL_REGISTRY[call.function.name]
        if asyncio.iscoroutinefunction(fn):
            return await fn(args)
        return await asyncio.to_thread(fn, args)
    return await asyncio.gather(*(run_one(c) for c in calls))

A 3-tool query (weather, news, translate) done in parallel is 3x faster than serial. Hiring managers grade this.

7. Reliability patterns (the boring expensive bits)

Production tool dispatch needs:

python

import time
from tenacity import retry, stop_after_attempt, wait_exponential

@retry(stop=stop_after_attempt(3), wait=wait_exponential(min=1, max=10))
def call_api(...): ...

def run_tool_safely(call):
    name = call.function.name
    try:
        args = TOOL_ARGS_BY_NAME[name](**json.loads(call.function.arguments))
    except (json.JSONDecodeError, ValidationError) as e:
        return {"error": f"bad args: {e}"}    # tell the LLM, do not crash

    if name not in TOOL_REGISTRY:
        return {"error": f"unknown tool {name}"}

    started = time.monotonic()
    try:
        result = TOOL_REGISTRY[name](args)
    except Exception as e:
        return {"error": str(e)[:200]}        # truncate so we do not blow context
    finally:
        latency_ms = (time.monotonic() - started) * 1000
        log_metric("tool_latency_ms", latency_ms, tool=name)

    return result

Key habits:

Never crash. Return an error field — the LLM can adapt.
Cap step count. Default 5-10. Hard stop on infinite loops.
Detect repeats. If the same tool with the same args was called twice with the same result, abort.
Idempotency keys for side-effecting tools (so retries do not double-charge a card).
Authorization checks inside each tool — never trust the LLM to enforce permissions.

8. Tool design rules from senior engineers

One verb per tool. create_invoice not do_invoice_stuff.
Make the description the contract. The LLM only knows what is in the description.
Return small, structured results. Big blobs blow up context.
Errors should be readable strings. "customer 42 not found" is better than a stack trace.
Side effects should be confirmable. Add a dry_run: bool arg or a separate confirm_* tool.
Mutable operations should require a reason. delete_record(id, reason: str) produces audit logs and forces the model to think.

Hands-on lab (4 hours)

Build a terminal travel agent:

Tools (define each with Pydantic args and a real or fake API):

search_flights(origin, destination, date) (use a CSV mock).
search_hotels(city, checkin, checkout, max_price) (CSV mock).
get_weather(city, date) (use Open-Meteo, free + no key).
convert_currency(from_, to_, amount) (use Frankfurter API).
web_search(query) (Tavily — free tier).

Agent loop:

Manual loop in OpenAI Responses API.
Max 8 steps.
Parallel tool calls when possible.
Retry on tool failure once, then return graceful error.

Acceptance criteria:

Asking "Find me a 3-day trip from Bengaluru to Goa under 25k INR including hotel and flight, weather-aware" returns a coherent plan with citations.
Logs every LLM call's tokens and every tool's latency.
100-line README with architecture diagram (mermaid).

Common pitfalls

Letting the LLM hand-write SQL/JSON without schema — always validate.
Tool descriptions that lie — "fast" when it is slow → model will time out and you will not know why.
Returning huge results — paginate or summarise.
Two tools with overlapping intents — model picks the wrong one. Merge or rename.
Forgetting to break the loop — if no tool call AND no final content, your code waits forever.

Self-check

What is the role of tool_choice="auto" vs tool_choice="required"?
How do you stop an infinite tool-call loop?
Why is parallel tool execution worth implementing?
What is an "idempotency key" and when do you need one?
Why must permission checks live inside the tool, not in the prompt?

Function calling — giving LLMs hands

Why this matters

Learning objectives

1. The mental model

2. Defining a tool

3. Manual loop with OpenAI Chat Completions

4. Manual loop with OpenAI Responses API

5. Anthropic's tool_runner (the easy mode)

6. Parallel tool calls

7. Reliability patterns (the boring expensive bits)

8. Tool design rules from senior engineers

Hands-on lab (4 hours)

Common pitfalls

Self-check

References

5. Anthropic's `tool_runner` (the easy mode)