# Chat Completions format

Source: https://platform.lyrouter.com/ai_gateway/docs/chat-completions/en
Last updated: 2026-10-19

Generates a model reply from the conversation history, with streaming and non-streaming responses, image input and tool calls. It is compatible with the OpenAI Chat Completions API: the official OpenAI SDKs and most OpenAI-compatible applications work once you set the Base URL to `https://api.lyrouter.com/v1`. The same model may be served by a channel that speaks the OpenAI-compatible protocol or the Anthropic protocol; the gateway converts requests and responses automatically, and this page also explains which parameters change during conversion.

`POST /v1/chat/completions`

## Authentication

Send your API key in a request header:

```text
Authorization: Bearer <API Key>
```

`Bearer` is case-sensitive. You can use `x-api-key: <API Key>` instead; when both are present, only `Authorization` is read. Create API keys on the workspace's **API Keys** page. Your API key is used only to authenticate with LYRouter and is never forwarded to the upstream provider. See [Base URLs and authentication](https://platform.lyrouter.com/ai_gateway/docs/call-gateway/en).

## Headers

| Header | Required | Description |
| --- | --- | --- |
| `Content-Type` | No | `application/json` is recommended. Without it, the gateway parses the request body as JSON; any other value (such as `application/x-www-form-urlencoded`) returns 400 |

## Request body

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `model` | string | No | Model ID, such as the system model identifier copied from the model marketplace; you can also use a virtual model `@vmodel/<name>` or `lyrouter/auto`. When it is omitted, `""` or `null`, the workspace's Default Model is used; without a Default Model, the request returns 400. See [Model identifiers and discovery](https://platform.lyrouter.com/ai_gateway/docs/model-names/en) |
| `messages` | array | Yes | Conversation messages; see the structure below |
| `stream` | boolean | No | When `true`, the response is streamed over SSE. Defaults to `false` |
| `stream_options` | object | No | For streaming requests the gateway always sets it to `{"include_usage": true}`, so the last chunk carries `usage`; any value you pass is overwritten |
| `max_tokens` | integer | No | Maximum number of tokens to generate in this call. Anthropic-protocol channels require it; when the request has neither `max_tokens` nor `max_completion_tokens`, the gateway adds 4096 |
| `max_completion_tokens` | integer | No | OpenAI's newer output limit field, passed through to OpenAI-compatible channels; for Anthropic-protocol channels it is used as `max_tokens` when `max_tokens` is absent |
| `temperature` | number | No | Sampling temperature. The valid range depends on the model (0–1 on Anthropic-protocol channels); the gateway does not rescale it |
| `top_p` | number | No | Nucleus sampling |
| `stop` | string \| array | No | Stop sequences; converted to `stop_sequences` for Anthropic-protocol channels |
| `tools` | array | No | Functions the model can call; see [Tool calls](https://platform.lyrouter.com/ai_gateway/docs/tool-call/en) |
| `tool_choice` | string \| object | No | `"auto"`, `"none"`, `"required"` or `{"type":"function","function":{"name":"..."}}`; for how these map across protocols, see [Tool calls](https://platform.lyrouter.com/ai_gateway/docs/tool-call/en) |
| `parallel_tool_calls` | boolean | No | Whether several tool calls may be returned at once; removed for Anthropic-protocol channels |
| `response_format` | object | No | `{"type":"json_object"}` or `{"type":"json_schema",...}`; takes effect only on OpenAI-compatible channels and is removed for Anthropic-protocol channels |
| `reasoning_effort` | string | No | Reasoning effort; the gateway handles it per provider, see "Reasoning parameters" below |
| `enable_thinking` / `thinking_budget` | boolean / integer | No | Thinking switch and thinking budget; the gateway handles them per provider, see "Reasoning parameters" below |
| `seed` | integer | No | Random seed; passed through to OpenAI-compatible channels, and whether it takes effect depends on the model; removed for Anthropic-protocol channels |
| `user` | string | No | End-user identifier, passed through to the upstream; whether it takes effect depends on the model |
| `frequency_penalty` / `presence_penalty` / `logprobs` / `n` | — | No | Passed through to OpenAI-compatible channels; whether they take effect depends on the model; removed for Anthropic-protocol channels |

Other parameters not listed in the table are forwarded unchanged to OpenAI-compatible channels; whether they take effect depends on the model.

### Structure of messages

Each message has a `role` and `content`:

| `role` | Purpose | Notes |
| --- | --- | --- |
| `system` | System prompt | Goes first. For Anthropic-protocol channels, all `system` and `developer` messages are merged into Anthropic's `system`; write their `content` as a string |
| `developer` | System prompt in newer OpenAI versions | Passed through to OpenAI-compatible channels; support depends on the model. Anthropic-protocol channels treat it as `system` |
| `user` | User input | `content` can be a string or an array of content blocks |
| `assistant` | The model's earlier replies | Put back unchanged in multi-turn conversations; carries `tool_calls` when it contains tool calls |
| `tool` | Tool result | Must carry `tool_call_id`, matching the call ID in the preceding assistant message |

`content` can be written in two ways. Write plain text directly as a string; to include images, use an array of content blocks:

```json
{
  "role": "user",
  "content": [
    { "type": "text", "text": "What is in this image?" },
    { "type": "image_url", "image_url": { "url": "https://example.com/cat.jpg" } }
  ]
}
```

- `image_url.url` can be a publicly accessible `https://` URL or Base64 data in the form `data:image/png;base64,<data>`.
- The model must support image input; text-only models return 400 `This model accepts text input only`. Use the "Input modalities" filter in the model marketplace to find suitable models; see [Multimodal capabilities](https://platform.lyrouter.com/ai_gateway/docs/multimodal/en).
- Anthropic-protocol channels convert only text blocks and image blocks; other content blocks such as `input_audio` and `file` work only on OpenAI-compatible channels.

### Reasoning parameters

The three reasoning parameters follow different providers' conventions. The gateway keeps or removes them according to the provider that actually serves the request, and does not convert one into another (requests to Anthropic-protocol channels are converted to `thinking`, as the table shows):

| Upstream that serves the request | `reasoning_effort` | `enable_thinking` | `thinking_budget` |
| --- | --- | --- | --- |
| OpenAI official API | Forwarded | Removed | Removed |
| Alibaba Cloud Model Studio (DashScope-compatible API) | Removed | Forwarded | Forwarded |
| DeepSeek official API | Removed | Removed | Removed (whether the model thinks depends on the selected model) |
| Anthropic-protocol channel | Removed | `true` without a budget becomes `thinking: {"type": "adaptive"}`; `false` leaves thinking off | Greater than 0 becomes `thinking: {"type": "enabled", "budget_tokens": N}` |
| Other OpenAI-compatible providers | Forwarded unchanged | Forwarded unchanged | Forwarded unchanged |

- The provider that served the request is shown in "Provider" in the [request log](https://platform.lyrouter.com/ai_gateway/docs/logs/en) details. When the same model has several sources, verify for each source whether the reasoning parameters take effect.
- When thinking is on for an Anthropic-protocol channel, the upstream requires `max_tokens` to be greater than `budget_tokens`. The gateway does not adjust these parameters; if the requirement is not met, the upstream returns 400.
- When a tool call is forced (`tool_choice` is `"required"` or a specified function) and the request has no `enable_thinking` / `thinking_budget`, Alibaba Cloud Model Studio channels automatically get `enable_thinking: false`, because these upstreams do not accept forced tool calls in thinking mode.
- The OpenAI Python SDK does not define `enable_thinking` or `thinking_budget`; pass them in `extra_body`, for example `extra_body={"enable_thinking": False}`. In the Node.js SDK, put them directly in the request parameters.

## Request examples

**Request examples**

cURL:
```bash
curl https://api.lyrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $LYROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Introduce LYRouter in one sentence."}
    ],
    "temperature": 0.7,
    "max_tokens": 512
  }'
```

Python:
```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LYROUTER_API_KEY"],
    base_url="https://api.lyrouter.com/v1",
)
completion = client.chat.completions.create(
    model="MODEL_ID",
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "Introduce LYRouter in one sentence."},
    ],
    temperature=0.7,
    max_tokens=512,
)
print(completion.choices[0].message.content)
print(completion.usage)
```

Node.js:
```js
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.LYROUTER_API_KEY,
  baseURL: "https://api.lyrouter.com/v1",
});
const completion = await client.chat.completions.create({
  model: "MODEL_ID",
  messages: [
    { role: "system", content: "You are a concise assistant." },
    { role: "user", content: "Introduce LYRouter in one sentence." },
  ],
  temperature: 0.7,
  max_tokens: 512,
});
console.log(completion.choices[0].message.content);
console.log(completion.usage);
```

## Response

A non-streaming request returns a `chat.completion` object; an error returns an `error` object. The 200 example below is a typical response from an OpenAI-compatible channel; the 400 is a request without `model` in a workspace that has no default model; the 429 means the API key has used up its daily token quota (see [429: rate limits and quotas](https://platform.lyrouter.com/ai_gateway/docs/rate-errors/en)).

**Response examples**

200:
```json
{
  "id": "chatcmpl-9f1c2a7e4b",
  "object": "chat.completion",
  "created": 1791950400,
  "model": "MODEL_ID",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "LYRouter is a trusted model service and token governance platform that lets you call many model providers with a single API key."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 1250,
    "completion_tokens": 32,
    "total_tokens": 1282,
    "prompt_tokens_details": { "cached_tokens": 1024 },
    "completion_tokens_details": { "reasoning_tokens": 0 }
  }
}
```

400:
```json
{
  "error": {
    "message": "you must provide a model parameter",
    "type": "invalid_request_error",
    "code": "missing_model"
  }
}
```

429:
```json
{
  "error": {
    "message": "rate limit exceeded on apikey total_tokens/day",
    "type": "rate_limit_error",
    "subject": "apikey",
    "metric": "total_tokens",
    "window": "day",
    "limit": 1000000,
    "used": 1000523,
    "reset_at": "2026-10-14T16:01:00Z"
  }
}
```

| Field | Description |
| --- | --- |
| `id` | Response ID generated by the upstream, not LYRouter's Request ID. The gateway does not return the Request ID in the response; copy it from the top of the log details when troubleshooting |
| `model` | Model name returned by the upstream, which may be written differently from the model ID in the request; the model actually used is the one shown in "Model (actual)" in the logs |
| `choices[0].message.content` | Answer text; may be `null` or an empty string when only tool calls are returned |
| `choices[0].message.tool_calls` | Tools the model asks to call; see [Tool calls](https://platform.lyrouter.com/ai_gateway/docs/tool-call/en) |
| `choices[0].message.reasoning_content` | Thinking process returned by some models; it also appears here when an Anthropic-protocol channel serves the request with thinking on |
| `choices[0].message.reasoning_blocks` | LYRouter extension field that appears only when thinking is on for an Anthropic-protocol channel, holding the original signed thinking blocks. In multi-turn tool calls, put the assistant message back unchanged so the gateway can hand the thinking blocks back to the upstream |
| `choices[0].finish_reason` | `stop` (normal end), `length` (output limit reached), `tool_calls` (waiting for tool results), `content_filter` (filtered by the upstream), and so on. For Anthropic-protocol channels, `end_turn` and `stop_sequence` map to `stop`, `max_tokens` maps to `length`, and `tool_use` maps to `tool_calls` |
| `usage` | Token usage for this call, reported by the upstream; see the table below |

### usage and the four token types in logs

The fields in `usage` depend on the upstream that served the request. They map to the four types under "Token usage" in the [request logs](https://platform.lyrouter.com/ai_gateway/docs/logs/en) as follows:

| Name in logs | OpenAI-compatible channel | Anthropic-protocol channel |
| --- | --- | --- |
| "Input (cache hit)" | `prompt_tokens_details.cached_tokens` (`prompt_cache_hit_tokens` for upstreams such as DeepSeek) | `cache_read_input_tokens` |
| "Input (cache miss)" | `prompt_tokens` minus cache hits | `prompt_tokens` |
| "Output" | `completion_tokens`, including reasoning tokens (`completion_tokens_details.reasoning_tokens`) | `completion_tokens` |
| "Explicit cache write" | Not present | `cache_creation_input_tokens` |

When an Anthropic-protocol channel serves the request, `usage` in the response looks like this, and `prompt_tokens` **does not include** cache hits or cache writes:

```json
"usage": {
  "prompt_tokens": 226,
  "completion_tokens": 32,
  "total_tokens": 258,
  "cache_read_input_tokens": 1024,
  "cache_creation_input_tokens": 0
}
```

If you estimate costs from `usage` yourself, first confirm which kind of upstream served the request, then price the four types separately; see [Estimating costs and cache pricing](https://platform.lyrouter.com/ai_gateway/docs/estimate/en). When reconciling, the logs are authoritative.

## Streaming

With `"stream": true`, the response `Content-Type` is `text/event-stream`, and each event is one `data: <JSON>` line followed by a blank line:

```text
data: {"id":"chatcmpl-9f1c2a7e4b","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-9f1c2a7e4b","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[{"index":0,"delta":{"content":"LYRouter is"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f1c2a7e4b","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[{"index":0,"delta":{"content":" a trusted model service and token governance platform."},"finish_reason":null}]}

data: {"id":"chatcmpl-9f1c2a7e4b","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-9f1c2a7e4b","object":"chat.completion.chunk","created":1791950400,"model":"MODEL_ID","choices":[],"usage":{"prompt_tokens":1250,"completion_tokens":32,"total_tokens":1282,"prompt_tokens_details":{"cached_tokens":1024}}}

data: [DONE]
```

- Answer text arrives piece by piece in `choices[0].delta.content`; the thinking process is in `delta.reasoning_content` (`delta.reasoning` on a few upstreams).
- The gateway always turns on `include_usage`, so just before the end there is a chunk whose `choices` is an empty array and which carries `usage`. Check that the array is not empty before reading `choices[0]`.
- `data: [DONE]` marks a normal end. If the connection closes before it arrives, or you receive an error event with `"code": "stream_interrupted"`, the output is incomplete; see [Timeouts and interrupted streams](https://platform.lyrouter.com/ai_gateway/docs/stream-errors/en).
- When an Anthropic-protocol channel serves the request, the gateway generates the chunks by conversion, and the chunk with `usage` has only three fields: `prompt_tokens`, `completion_tokens` and `total_tokens`. Check cache counts in the logs.

**Streaming examples**

cURL:
```bash
curl -N https://api.lyrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $LYROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"MODEL_ID","stream":true,"messages":[{"role":"user","content":"Write a two-line poem about autumn"}]}'
```

Python:
```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LYROUTER_API_KEY"],
    base_url="https://api.lyrouter.com/v1",
)
stream = client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Write a two-line poem about autumn"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)
```

Node.js:
```js
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.LYROUTER_API_KEY,
  baseURL: "https://api.lyrouter.com/v1",
});
const stream = await client.chat.completions.create({
  model: "MODEL_ID",
  messages: [{ role: "user", content: "Write a two-line poem about autumn" }],
  stream: true,
});
for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  if (chunk.usage) console.log("\n", chunk.usage);
}
```

For a comparison of event formats across the three chat endpoints and key points for client implementations, see [Streaming](https://platform.lyrouter.com/ai_gateway/docs/streaming/en).

## Gateway behavior

Compared with calling the upstream directly, calling through LYRouter differs in these ways:

- **Channel selection and fallback**: the gateway selects a channel for the model according to the workspace's routing strategy, moves to the next channel on failure, and may finally switch to the workspace's Default Model; see [Retries and fallback](https://platform.lyrouter.com/ai_gateway/docs/fallback/en) and [Channels and routing](https://platform.lyrouter.com/ai_gateway/docs/routing/en). Once streaming output has started, the gateway no longer switches channels.
- **Protocol conversion**: when the channel uses the Anthropic protocol, the gateway converts the request to Anthropic Messages format and converts the response back to Chat Completions format. `response_format`, `seed`, `n`, `logprobs`, `top_logprobs`, `frequency_penalty`, `presence_penalty`, `logit_bias`, `parallel_tool_calls` and `reasoning_effort` are removed, and `metadata` keeps only `user_id`; these removals are not reported in response headers. For a comparison, see [Capabilities and compatibility](https://platform.lyrouter.com/ai_gateway/docs/capabilities/en).
- **Reasoning parameters**: kept, removed or converted according to the table in "Reasoning parameters" above.
- **Prompt cache markers**: `cache_control` takes effect only on Anthropic-protocol channels, where markers on text content blocks and tool definitions are kept. With `"ttl": "1h"`, the gateway sets the `anthropic-beta` request header to the value required for the 1-hour cache (overriding any value the client sent). For OpenAI-compatible channels, all markers are removed and the upstream's implicit caching does the work.
- **Extension fields in history messages**: before sending to OpenAI-compatible channels, the gateway removes `reasoning_content`, `reasoning_blocks` and `cache_control` from messages.
- **Fixed JSON key order**: the gateway reorders the keys of every JSON object in the request body sent upstream into a fixed order, including JSON inside tool call argument strings, so that the same content always produces the same bytes and the upstream prefix cache hits more often; see [Reducing costs and improving cache reuse](https://platform.lyrouter.com/ai_gateway/docs/cost-practice/en).
- **Number precision**: the gateway parses and re-encodes the request body, and numbers keep at most 14 significant digits. Longer integers (such as 19-digit IDs) become scientific notation; pass them as strings.
- **Usage**: streaming requests always return `usage`; the fields in `usage` vary by upstream, see the mapping table above.

## Common errors

Error responses take the form `{"error":{"message":"...","type":"...","code":"..."}}`; only some errors include `code`.

| Status code | `error.message` | `error.code` | Cause and fix |
| --- | --- | --- | --- |
| 400 | `you must provide a model parameter` | `missing_model` | No `model` was given and the workspace has no Default Model. Specify a model ID. Some versions also return this error when the request body is not a valid JSON object; see the next row |
| 400 | `request body must be a JSON object` | `invalid_json` | The request body is not a valid JSON object, for example the JSON is truncated or the top level is an array. Some versions return 400 `you must provide a model parameter` (`missing_model`) instead. When the workspace has a Default Model, the response is `{"message":"could not get parse JSON request body: <parse error>"}`, with no outer `error` |
| 400 | `This model accepts text input only` | `unsupported_model_input` | Non-text content such as images was sent to a text-only model; use a model that supports image input |
| 400 | `This model does not support the chat_completions API` | `unsupported_model_api` | The model does not support the chat endpoint, for example an image generation model |
| 403 | `Model is not available in the model plaza` | `model_not_available` | The workspace has not enabled this model. Ask an administrator to enable it in the model marketplace, or use a model that is enabled |
| 503 | `all routed candidates exhausted` | `no_route_candidate` | No channel can serve this request, commonly because `@vmodel/<name>` is misspelled or the model has no available source; see [Unavailable models](https://platform.lyrouter.com/ai_gateway/docs/unavailable/en) |

- For 401 / 403 authentication issues, see [401 / 403: authentication and access](https://platform.lyrouter.com/ai_gateway/docs/auth-errors/en); for 429, see [429: rate limits and quotas](https://platform.lyrouter.com/ai_gateway/docs/rate-errors/en).
- 500 / 504 mean the connection to the upstream failed or the wait for the upstream timed out; the response body is an HTML page rather than JSON. See [Timeouts and interrupted streams](https://platform.lyrouter.com/ai_gateway/docs/stream-errors/en).
- Some 400s generated by the gateway itself are a single line of plain text rather than JSON, for example `unsupported content-type: …` when `Content-Type` is not `application/json`. Their response header is currently still `Content-Type: application/json`, so check whether the body is JSON before parsing the error.
- Errors returned by the upstream provider are forwarded unchanged; the status code and error text come from the upstream. For the full list, see [Errors and request identifiers](https://platform.lyrouter.com/ai_gateway/docs/errors/en).