# Tool calls

Source: https://platform.lyrouter.com/ai_gateway/docs/tool-call/en
Last updated: 2026-10-22

Tool calling (function calling) lets the model request a call to a function you define, with structured arguments: your application declares tools in the request, the model returns the function name and arguments to call, your application runs the function itself and sends the result back, and the model gives the final answer. LYRouter only forwards requests and converts between protocols; it never runs any of your business operations. Chat Completions, Responses and Anthropic Messages structure tools differently, and the gateway converts automatically between Chat Completions and the Anthropic protocol. This chapter uses Chat Completions for the complete examples and explains what happens during conversion.

## How it works

1. Define functions in the request's `tools`: name, description of purpose, and the JSON Schema of the parameters.
2. When the model decides to call a tool, the response's `message.tool_calls` gives each call's `id`, function name and arguments (a JSON string), and `finish_reason` is usually `tool_calls`.
3. Your application parses the arguments, validates them and runs the function.
4. Append the assistant message returned by the model to `messages` unchanged, then append one `role: tool` message for each call, with `tool_call_id` set to that call's `id`.
5. Send the request again. The model answers based on the tool results, or calls more tools, until the response no longer contains `tool_calls`.

To decide whether to run tools, check whether the response contains `tool_calls` (`tool_use` blocks in Anthropic Messages) rather than relying only on `finish_reason` (`stop_reason`): for example, when `tool_choice` specifies a function, Alibaba Cloud Model Studio may return `tool_calls` with `finish_reason` `stop`; and through Anthropic Messages, specifying a tool with `{"type": "tool"}` currently returns a `tool_use` block but with `stop_reason` `end_turn` (`any` and `auto` behave normally). The example code below checks `message.tool_calls`.

## Defining tools

```json
{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather for a city. Call this when the user asks about weather or temperature.",
    "parameters": {
      "type": "object",
      "properties": {
        "city": { "type": "string", "description": "City name, for example Beijing" },
        "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
      },
      "required": ["city"]
    }
  }
}
```

| Field | Description |
| --- | --- |
| `type` | Always `function` |
| `function.name` | Function name, using only letters, digits, underscores and hyphens; must be unique within a request |
| `function.description` | When to call the function and what it returns. The more specific the description, the more easily the model picks the right tool |
| `function.parameters` | JSON Schema of the parameters, with `object` at the top level. Define long integer IDs as `string`; for why, see "Cross-protocol conversion" below |
| `function.strict` | Whether arguments are generated strictly according to the schema; passed through to OpenAI-compatible channels, and whether it takes effect depends on the model; not forwarded to Anthropic-protocol channels |

The model must support tool calling; check this in the model's details in the model marketplace.

## A complete two-turn example

### Turn 1: send the question and tool definition

```bash
curl https://api.lyrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $LYROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "messages": [
      {"role": "user", "content": "What is the weather like in Beijing right now?"}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'
```

The model decides to call the tool and returns:

```json
{
  "id": "chatcmpl-5d1e8b20a3",
  "object": "chat.completion",
  "created": 1791950400,
  "model": "MODEL_ID",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_8f2a1c",
            "type": "function",
            "function": { "name": "get_weather", "arguments": "{\"city\":\"Beijing\"}" }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": { "prompt_tokens": 96, "completion_tokens": 18, "total_tokens": 114 }
}
```

`arguments` is a JSON **string** that you need to parse yourself. `content` may be `null` or an empty string, or it may contain some explanatory text.

### Turn 2: send back the tool result

Put the previous turn's assistant message back unchanged, followed by a `role: tool` message. Keep `tools` the same as in turn 1:

```bash
curl https://api.lyrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $LYROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "messages": [
      {"role": "user", "content": "What is the weather like in Beijing right now?"},
      {"role": "assistant", "content": null, "tool_calls": [
        {"id": "call_8f2a1c", "type": "function",
         "function": {"name": "get_weather", "arguments": "{\"city\":\"Beijing\"}"}}
      ]},
      {"role": "tool", "tool_call_id": "call_8f2a1c",
       "content": "{\"city\":\"Beijing\",\"condition\":\"Sunny\",\"temperature_c\":22}"}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'
```

The model answers based on the tool result:

```json
{
  "id": "chatcmpl-5d1e8b20b7",
  "object": "chat.completion",
  "created": 1791950403,
  "model": "MODEL_ID",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "It is sunny in Beijing right now, at 22°C." },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 142, "completion_tokens": 14, "total_tokens": 156 }
}
```

When several `tool_calls` are returned at once, append one `role: tool` message for each call after the assistant message, then send the next request.

## A minimal runnable loop

The program below turns the two turns above into a loop: it runs the tools the model requests until the model gives a final answer, for at most 5 turns. To install the SDK and set environment variables, see [Your first model call](https://platform.lyrouter.com/ai_gateway/docs/quickstart/en).

**Tool call loop**

Python:
```python
import json
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LYROUTER_API_KEY"],
    base_url="https://api.lyrouter.com/v1",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]

def get_weather(city):
    # Replace with a real weather lookup
    return {"city": city, "condition": "Sunny", "temperature_c": 22}

TOOL_FUNCTIONS = {"get_weather": get_weather}

messages = [{"role": "user", "content": "What is the weather like in Beijing right now?"}]

for _ in range(5):  # Limit the number of turns
    response = client.chat.completions.create(
        model="MODEL_ID", messages=messages, tools=tools
    )
    message = response.choices[0].message
    if not message.tool_calls:
        print(message.content)
        break

    # Append the assistant message unchanged (including tool_calls)
    messages.append(message.model_dump(exclude_none=True))
    for call in message.tool_calls:
        func = TOOL_FUNCTIONS.get(call.function.name)
        try:
            args = json.loads(call.function.arguments or "{}")
            result = func(**args) if func else {"error": "unknown tool"}
        except Exception as exc:  # Invalid arguments or failed execution: tell the model about the error
            result = {"error": str(exc)}
        messages.append(
            {
                "role": "tool",
                "tool_call_id": call.id,
                "content": json.dumps(result, ensure_ascii=False),
            }
        )
```

Node.js:
```js
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.LYROUTER_API_KEY,
  baseURL: "https://api.lyrouter.com/v1",
});

const tools = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather for a city.",
      parameters: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"],
      },
    },
  },
];

function getWeather({ city }) {
  // Replace with a real weather lookup
  return { city, condition: "Sunny", temperature_c: 22 };
}

const toolFunctions = { get_weather: getWeather };
const messages = [{ role: "user", content: "What is the weather like in Beijing right now?" }];

for (let turn = 0; turn < 5; turn++) {
  const response = await client.chat.completions.create({
    model: "MODEL_ID",
    messages,
    tools,
  });
  const message = response.choices[0].message;
  if (!message.tool_calls?.length) {
    console.log(message.content);
    break;
  }

  messages.push(message); // Append the assistant message unchanged (including tool_calls)
  for (const call of message.tool_calls) {
    const fn = toolFunctions[call.function.name];
    let result;
    try {
      const args = JSON.parse(call.function.arguments || "{}");
      result = fn ? fn(args) : { error: "unknown tool" };
    } catch (err) {
      result = { error: String(err) }; // Invalid arguments or failed execution: tell the model about the error
    }
    messages.push({
      role: "tool",
      tool_call_id: call.id,
      content: JSON.stringify(result),
    });
  }
}
```

## tool_choice and parallel calls

| `tool_choice` | Meaning | Sent to an Anthropic-protocol channel as |
| --- | --- | --- |
| Omitted, or `"auto"` | The model decides whether to call a tool | `{"type": "auto"}` |
| `"required"` | The model must call at least one tool | `{"type": "any"}` |
| `{"type": "function", "function": {"name": "get_weather"}}` | The model must call the specified tool | `{"type": "tool", "name": "get_weather"}` |
| `"none"` | Do not call tools | Not forwarded, so the model may still call tools. If you do not want tool calls, do not send `tools` |

- `parallel_tool_calls: false` asks for at most one call at a time. It is passed through to OpenAI-compatible channels, and whether it takes effect depends on the model; it is removed for Anthropic-protocol channels, where the model may still return several calls at once. Always handle `tool_calls` as an array in your application.
- When a call is forced (`"required"` or a specified function) and the request has no `enable_thinking` / `thinking_budget`, Alibaba Cloud Model Studio channels automatically get `enable_thinking: false`, because these upstreams do not accept forced tool calls in thinking mode.

## Assembling tool arguments when streaming

In a streaming request, a tool call arrives across several chunks: the first fragment carries the `id`, `type` and function name, and later fragments carry only a piece of `arguments`.

```text
data: {"id":"chatcmpl-5d1e8b20a3","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[{"index":0,"delta":{"role":"assistant","content":null,"tool_calls":[{"index":0,"id":"call_8f2a1c","type":"function","function":{"name":"get_weather","arguments":""}}]},"finish_reason":null}]}

data: {"id":"chatcmpl-5d1e8b20a3","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"city\":"}}]},"finish_reason":null}]}

data: {"id":"chatcmpl-5d1e8b20a3","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"Beijing\"}"}}]},"finish_reason":null}]}

data: {"id":"chatcmpl-5d1e8b20a3","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}

data: {"id":"chatcmpl-5d1e8b20a3","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[],"usage":{"prompt_tokens":96,"completion_tokens":18,"total_tokens":114}}

data: [DONE]
```

- Group fragments by `tool_calls[].index`, and join the `arguments` that share an `index` in order of arrival to get the complete string.
- Take `id` and the function name from the first non-empty value. Some upstreams repeat `id` and `name` as empty strings in later fragments; do not overwrite earlier values with empty ones.
- Parse the arguments JSON only after you receive the chunk with `finish_reason` or the stream ends; the fragments in between are not valid JSON. `finish_reason` is not necessarily `"tool_calls"`; see "How it works" above.
- In Anthropic Messages the equivalents are `content_block_start` (a `tool_use` block with `id` and `name`) and several `input_json_delta` events; join `partial_json` by the block's `index`. In Responses they are `response.output_item.added` (a `function_call` item) and `response.function_call_arguments.delta`, with `response.function_call_arguments.done` giving the complete arguments. For event formats, see [Streaming](https://platform.lyrouter.com/ai_gateway/docs/streaming/en).

## Structure across the three protocols

| Step | Chat Completions | Responses | Anthropic Messages |
| --- | --- | --- | --- |
| Define tools | `{"type": "function", "function": {"name", "description", "parameters"}}` | `{"type": "function", "name", "description", "parameters"}`, without the `function` wrapper | `{"name", "description", "input_schema"}` |
| Model calls a tool | `message.tool_calls[]`: `id`, `function.name`, `function.arguments` (JSON string) | Items in `output[]` with `type: "function_call"`: `call_id`, `name`, `arguments` (JSON string) | Blocks in `content[]` with `type: "tool_use"`: `id`, `name`, `input` (JSON object) |
| Signals that tool results are needed | `message.tool_calls` is not empty (`finish_reason` is usually `"tool_calls"`) | `output` contains a `function_call` item | `content` contains a `tool_use` block (`stop_reason` is usually `"tool_use"`) |
| Send results back | `{"role": "tool", "tool_call_id": "...", "content": "..."}` | Append `{"type": "function_call_output", "call_id": "...", "output": "..."}` to `input` | Put a `{"type": "tool_result", "tool_use_id": "...", "content": "..."}` block in a `role: "user"` message |
| Matching | `tool_call_id` = `tool_calls[].id` | `call_id` = `function_call.call_id` | `tool_use_id` = `tool_use.id` |
| Force a call | `"required"` / specify `function.name` | `"required"` / `{"type": "function", "name": "..."}` | `{"type": "any"}` / `{"type": "tool", "name": "..."}` |
| Disable parallel calls | `parallel_tool_calls: false` | `parallel_tool_calls: false` | `tool_choice.disable_parallel_tool_use: true` |

A call and its returned result in Responses:

```json
{"type": "function_call", "id": "fc_68f1b0a1", "call_id": "call_8f2a1c", "name": "get_weather", "arguments": "{\"city\":\"Beijing\"}", "status": "completed"}
{"type": "function_call_output", "call_id": "call_8f2a1c", "output": "{\"condition\":\"Sunny\",\"temperature_c\":22}"}
```

A call and its returned result in Anthropic Messages:

```json
{"role": "assistant", "content": [{"type": "tool_use", "id": "toolu_01A09q90qw90lq917835lq9", "name": "get_weather", "input": {"city": "Beijing"}}]}
{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_01A09q90qw90lq917835lq9", "content": "{\"condition\":\"Sunny\",\"temperature_c\":22}"}]}
```

The gateway forwards Responses requests unchanged and does not convert their tool structure. When you continue with `previous_response_id`, the gateway uses it only to identify the session, and the continuation request is not necessarily sent to the upstream that created the previous response; see [Responses](https://platform.lyrouter.com/ai_gateway/docs/responses-api/en).

## Cross-protocol conversion

When a Chat Completions request is served by an Anthropic-protocol channel, or an Anthropic Messages request is served by an OpenAI-compatible channel, the gateway converts the tool-related structures:

| Direction | What the gateway does |
| --- | --- |
| Chat Completions → Anthropic-protocol channel | `tools[].function` becomes `name`, `description` and `input_schema`; only tools with `type: "function"` are converted, and `strict` is not forwarded. `tool_calls[].function.arguments` in the history is parsed into an `input` object, and sent as `{}` when it is not valid JSON. Each `role: tool` message becomes a user message with a `tool_result` block. `tool_choice` is converted as in the table above, and `parallel_tool_calls` is removed. In the response, `tool_use` blocks are converted back to `tool_calls`, and `stop_reason: "tool_use"` becomes `finish_reason: "tool_calls"` |
| Anthropic Messages → OpenAI-compatible channel | `tools` become tools with `type: "function"`; `tool_use` blocks become `tool_calls`, with `input` serialized into the `arguments` string; `tool_result` blocks become `role: tool` messages, and `is_error` is not forwarded. `auto`, `any` and `tool` in `tool_choice` become `"auto"`, `"required"` and a specified function respectively, while `{"type": "none"}` and `disable_parallel_tool_use` are not forwarded. In the response, `tool_calls` are converted back to `tool_use` blocks, and `input` is `{}` when the arguments are not valid JSON; `stop_reason` is converted from the upstream's `finish_reason` (`tool_calls` becomes `tool_use`, `stop` becomes `end_turn`), regardless of whether `tool_calls` are present |

Before sending a request, the gateway also applies the following:

- **Fixed JSON key order**: the gateway reorders the keys of every JSON object in the request body sent upstream into a fixed order (array order is unchanged), and does the same for the JSON inside the `tool_calls[].function.arguments` strings in history messages and the `arguments` strings of `function_call` items in a Responses `input`. When the client writes the same argument object with a different key order on each turn, the upstream still sees the same bytes, so the prompt cache is not invalidated by it. Arguments that are not valid JSON objects, or that contain 15 or more consecutive digits, are left as they are. This affects only the request sent upstream, not the content returned to you.
- **Number precision**: when the gateway parses and re-encodes JSON, numbers keep at most 14 significant digits. The `input` objects of Anthropic Messages, and tool arguments during cross-protocol conversion, are all parsed, so long integers such as 19-digit order numbers or Snowflake IDs become scientific notation; define them as `string` in the parameter schema.
- **Thinking content**: before sending to OpenAI-compatible channels, the gateway removes `reasoning_content` from history assistant messages. The DeepSeek family is the exception: its thinking mode requires the current turn's reasoning to be passed back, so non-empty `reasoning_content` on assistant messages after the last user message is kept; if DeepSeek still returns a 400 because it is missing, the gateway fills in an empty value and retries once on the same channel. When an Anthropic-protocol channel with thinking on serves the request, the `message` of a non-streaming response carries `reasoning_blocks` (a LYRouter extension field); put the assistant message back unchanged so the gateway can hand the signed thinking blocks back to the upstream.

## Notes

- **Send tool results and extra instructions separately**: when calling through Anthropic Messages, put the `tool_result` block and any additional text for the model in two separate messages rather than the same user message, so that both are kept intact when converted for an OpenAI-compatible channel.
- **Report failures too**: when a tool fails, write the error into the result content (for example `{"error": "City not found"}`) so the model can adjust; the `is_error` flag of `tool_result` is not kept when converting for an OpenAI-compatible channel.
- **Validate before running**: arguments generated by the model may be incomplete or invalid. Validate them against the schema before running the function, and check that the caller is allowed to perform the operation. For tools with side effects (placing orders, sending email, deleting data), ask the user to confirm first, and carry an idempotency key with each operation so that a retry after an interrupted stream does not run it twice.
- **Limit the number of turns**: set a maximum number of turns and an overall timeout for the tool loop, so the model cannot keep calling tools indefinitely.
- **Keep tool definitions stable**: within one session, do not add, remove, reorder or modify tools; otherwise no later prompt cache can hit. See [Reducing costs and improving cache reuse](https://platform.lyrouter.com/ai_gateway/docs/cost-practice/en).
- **Logs keep tool calls**: tool calls returned by the model (function names and arguments) are stored even when content logging is off, so do not put passwords or other sensitive information in arguments; see [Logs and data processing](https://platform.lyrouter.com/ai_gateway/docs/privacy/en).

## FAQ

### Why did the model answer directly without calling a tool?

**Answer:** First confirm that the selected model supports tool calling, then check whether the tool's `description` says when it should be called. To make sure a tool is called, set `tool_choice` to `"required"` or to a specific function.

### Why does the upstream return 400 when I force a tool call?

**Answer:** Some models do not accept forced tool calls in thinking mode; for example, Alibaba Cloud Model Studio returns `The tool_choice parameter does not support being set to required or object in thinking mode`. When you do not set thinking parameters, the gateway turns off thinking automatically for Alibaba Cloud Model Studio channels; if your request sets `enable_thinking: true` or `thinking_budget` itself, remove it, or change `tool_choice` to `"auto"`.

### Why does the turn 2 request fail with an error about a missing tool call ID or wrong message order?

**Answer:** Check three things: whether the assistant message was put back unchanged with its `tool_calls`; whether every call has a `role: tool` message whose `tool_call_id` matches the call's `id`; and whether the `role: tool` messages directly follow their assistant message. The error text comes from the upstream; view it under "Upstream response" in the "Raw packets" tab of the [request log](https://platform.lyrouter.com/ai_gateway/docs/logs/en) details.

### How do I see tool calls in the logs?

**Answer:** Open the "Conversation" tab of the [request log](https://platform.lyrouter.com/ai_gateway/docs/logs/en) details. "Tool calls" lists the tools and `tool_choice` defined in the request, and the function name and arguments of each call the model returned. To confirm which provider the request was routed to and whether protocol conversion happened, check "Provider" and "Upstream request" under "Raw packets".