# Responses format

Source: https://platform.lyrouter.com/ai_gateway/docs/responses-api/en
Last updated: 2026-10-22

Creates a model response: you pass text or a list of input items in `input`, and it returns a response object with an `id`, with the output in the `output` array, where each item is a message, a function call or reasoning content; streaming is supported. It is compatible with the OpenAI Responses API: the OpenAI SDK's `client.responses` and clients that use the Responses protocol (such as the [Codex](https://platform.lyrouter.com/ai_gateway/docs/codex/en) CLI) work once you set the Base URL to `https://api.lyrouter.com/v1`, the same as for Chat Completions. The gateway forwards requests in the Responses protocol without protocol conversion, so the selected model and the channel serving it must both support Responses.

`POST /v1/responses`

## Authentication

Send your API key in a request header:

```text
Authorization: Bearer <API Key>
```

`Bearer` is case-sensitive. You can use `x-api-key: <API Key>` instead; when both are present, only `Authorization` is read. Create API keys on the workspace's **API Keys** page. Your API key is used only to authenticate with LYRouter and is never forwarded to the upstream provider. See [Base URLs and authentication](https://platform.lyrouter.com/ai_gateway/docs/call-gateway/en).

## Headers

| Header | Required | Description |
| --- | --- | --- |
| `Content-Type` | No | `application/json` is recommended. Without it, the gateway parses the request body as JSON; any other value (such as `application/x-www-form-urlencoded`) returns 400 |

## Request body

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `model` | string | No | Model ID; you can also write `@vmodel/<name>` ([virtual model](https://platform.lyrouter.com/ai_gateway/docs/presets/en)) or `lyrouter/auto` ([Auto routing](https://platform.lyrouter.com/ai_gateway/docs/auto/en)). When it is omitted, `""` or `null`, the workspace's Default Model is used; without a Default Model, the request returns 400 |
| `input` | string / array | Yes | A string is equivalent to one user message; an array holds message items and other input items in order, see "Items in the input array" below |
| `instructions` | string | No | System-level instructions. With a virtual model, the virtual model's system prompt is appended after these instructions |
| `stream` | boolean | No | When `true`, the response is returned as an SSE event stream; see "Streaming" below |
| `tools` | array | No | Function tools are written as `{"type": "function", "name", "description", "parameters", "strict"}`, with the fields flat rather than wrapped in `function` as in Chat. Built-in tools (such as `web_search`) are forwarded as is; whether they work depends on the upstream |
| `tool_choice` | string / object | No | `auto`, `none`, `required`, or `{"type": "function", "name": "..."}` |
| `parallel_tool_calls` | boolean | No | Whether several function calls may be returned at once |
| `max_output_tokens` | integer | No | Output limit, including reasoning tokens. Use this field; Chat's `max_tokens` is forwarded as is, and whether it takes effect is up to the upstream |
| `reasoning` | object | No | For example `{"effort": "low", "summary": "auto"}`. Passed through to the upstream; whether it takes effect depends on the model |
| `text.format` | object | No | Output format: `{"type": "text"}`, `{"type": "json_object"}`, or `{"type": "json_schema", "name": "...", "schema": {...}, "strict": true}`. The effect depends on the upstream: for example, some upstreams still wrap the JSON in a Markdown json code block with `json_object`, and do not follow the schema with `json_schema` (the `text.format` echoed in the response names the field `schema_`). Verify with the selected model before relying on structured output, and strip code block markers before parsing |
| `temperature`, `top_p` | number | No | Passed through to the upstream; reasoning models usually do not support them |
| `store` | boolean | No | Whether the upstream stores this response. The gateway forwards it as is; when it is omitted, the upstream default applies (`true` for OpenAI) |
| `previous_response_id` | string | No | Continues from the previous response, with the upstream assembling the context; see "Multi-turn conversations and function calls" below |
| `include` | array | No | Extra content to return, for example `["reasoning.encrypted_content"]`. Passed through |
| `metadata` | object | No | Passed through |

### Items in the input array

| Item | Format | Purpose |
| --- | --- | --- |
| Message | `{"role": "user", "content": "..."}`. `role` can be `user`, `assistant`, `system` or `developer`; `content` can be a string or an array of content parts such as `input_text` and `input_image` | Passes conversation content. Image input requires model support; see [Multimodal capabilities](https://platform.lyrouter.com/ai_gateway/docs/multimodal/en) |
| Function call result | `{"type": "function_call_output", "call_id": "...", "output": "..."}` | Returns a tool's result to the model |
| Output items from the previous turn | The `message`, `function_call` and `reasoning` items from the previous response's `output`, put back unchanged | Lets the client carry the history itself when it does not use `previous_response_id` |

## Request examples

**Request examples**

cURL:
```bash
curl https://api.lyrouter.com/v1/responses \
  -H "Authorization: Bearer $LYROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "instructions": "You are a concise assistant.",
    "input": "Explain what an API gateway is in one sentence.",
    "max_output_tokens": 1024
  }'
```

Python:
```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LYROUTER_API_KEY"],
    base_url="https://api.lyrouter.com/v1",
)
response = client.responses.create(
    model="MODEL_ID",
    instructions="You are a concise assistant.",
    input="Explain what an API gateway is in one sentence.",
    max_output_tokens=1024,
)
print(response.output_text)
print(response.usage)
```

Node.js:
```js
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.LYROUTER_API_KEY,
  baseURL: "https://api.lyrouter.com/v1",
});
const response = await client.responses.create({
  model: "MODEL_ID",
  instructions: "You are a concise assistant.",
  input: "Explain what an API gateway is in one sentence.",
  max_output_tokens: 1024,
});
console.log(response.output_text);
console.log(response.usage);
```

Replace `MODEL_ID` with a model ID that is enabled in your workspace and supports Responses. `output_text` is a convenience property of the SDK that joins all `output_text` content; the raw JSON has no such field, so iterate over `output` when you parse the HTTP response directly. The Python example requires openai Python SDK 1.66 or later; older versions have no `client.responses` and raise `AttributeError: 'OpenAI' object has no attribute 'responses'`.

## Response

A non-streaming request returns a `response` object; an error returns an `error` object. The 200 example below is a typical response with a reasoning summary; the 400 means the selected model does not support Responses; the 403 means the workspace has not enabled this model.

**Response examples**

200:
```json
{
  "id": "resp_68ee1b9a52c48190a71c3e7d6a0b2f5e",
  "object": "response",
  "created_at": 1760427600,
  "status": "completed",
  "error": null,
  "incomplete_details": null,
  "model": "gpt-5.5",
  "instructions": "You are a concise assistant.",
  "output": [
    {
      "id": "rs_68ee1b9ab1d08190b3a2c5e7f9013d4a",
      "type": "reasoning",
      "summary": [
        { "type": "summary_text", "text": "The user wants a one-sentence definition, so give the conclusion directly." }
      ]
    },
    {
      "id": "msg_68ee1b9b0e7c8190b9f2d4a6c8e01a37",
      "type": "message",
      "status": "completed",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "An API gateway is the single entry point between clients and backend services, handling authentication, routing, rate limiting and monitoring.",
          "annotations": []
        }
      ]
    }
  ],
  "max_output_tokens": 1024,
  "previous_response_id": null,
  "reasoning": { "effort": "low", "summary": "auto" },
  "store": true,
  "text": { "format": { "type": "text" } },
  "tool_choice": "auto",
  "tools": [],
  "usage": {
    "input_tokens": 1830,
    "input_tokens_details": { "cached_tokens": 1536 },
    "output_tokens": 196,
    "output_tokens_details": { "reasoning_tokens": 128 },
    "total_tokens": 2026
  },
  "metadata": {}
}
```

400:
```json
{
  "error": {
    "message": "This model does not support the responses API",
    "type": "invalid_request_error",
    "code": "unsupported_model_api"
  }
}
```

403:
```json
{
  "error": {
    "message": "Model is not available in the model plaza",
    "type": "access_denied",
    "code": "model_not_available"
  }
}
```

| Field | Description |
| --- | --- |
| `id` | Response ID generated by the upstream, commonly starting with `resp_` (some upstreams return IDs starting with `resp-`); it is not LYRouter's Request ID. Used for `previous_response_id` |
| `status` | `completed`; `incomplete` when the output reaches `max_output_tokens`, with the reason in `incomplete_details.reason`; it can also be `failed` or `cancelled`, and `queued` or `in_progress` for background runs |
| `model` | The actual model name returned by the upstream, which can differ from the model ID in the request. Corresponds to "Model (actual)" in the logs |
| `output` | Array of output items; handle each one by `type`: a `message` item's `content` holds `output_text`; `function_call` is a function call; `reasoning` is a reasoning summary or encrypted reasoning content. Do not parse it the Chat way through `choices[0].message` |
| `usage` | Token usage for this call. The gateway returns the upstream's values unchanged; see the table below |

### usage and the four token types in logs

How `usage` maps to the four types under "Token usage" in the [request logs](https://platform.lyrouter.com/ai_gateway/docs/logs/en):

| `usage` field | Meaning | Item in the logs |
| --- | --- | --- |
| `input_tokens` | All input, **including** the part that hit the cache | "Input (cache miss)" + "Input (cache hit)" |
| `input_tokens_details.cached_tokens` | Input that hit the upstream prompt cache | "Input (cache hit)" |
| `output_tokens` | All output, **including** reasoning tokens | "Output" |
| `output_tokens_details.reasoning_tokens` | The part of the output used for reasoning | Included in "Output" |

In the example above, "Input (cache miss)" is 1830 − 1536 = 294, "Input (cache hit)" is 1536 and "Output" is 196; Responses upstreams do not produce "Explicit cache write". For billing, see [Estimating costs and cache pricing](https://platform.lyrouter.com/ai_gateway/docs/estimate/en).

## Streaming

With `"stream": true`, the response is `text/event-stream`. Each event consists of an `event:` line and a `data:` line; `type` in `data` matches the event name, and `sequence_number` increases. In the actual stream each event is also preceded by an `id:` line, and there may be no space after the colon in `event:` and `data:` (as in `data:{...}`); both forms are valid under the SSE specification, so handle both if you write your own parser. The fragment below omits the `id:` lines and some fields of the `response` object:

```text
event: response.created
data: {"type":"response.created","sequence_number":0,"response":{"id":"resp_68ee1b9a52c48190a71c3e7d6a0b2f5e","object":"response","status":"queued","output":[],"usage":null}}

event: response.output_item.added
data: {"type":"response.output_item.added","sequence_number":2,"output_index":0,"item":{"id":"msg_68ee1b9b0e7c8190b9f2d4a6c8e01a37","type":"message","status":"in_progress","role":"assistant","content":[]}}

event: response.output_text.delta
data: {"type":"response.output_text.delta","sequence_number":4,"item_id":"msg_68ee1b9b0e7c8190b9f2d4a6c8e01a37","output_index":0,"content_index":0,"delta":"An API gateway is"}

event: response.output_text.done
data: {"type":"response.output_text.done","sequence_number":9,"item_id":"msg_68ee1b9b0e7c8190b9f2d4a6c8e01a37","output_index":0,"content_index":0,"text":"An API gateway is the single entry point between clients and backend services, handling authentication, routing, rate limiting and monitoring."}

event: response.completed
data: {"type":"response.completed","sequence_number":12,"response":{"id":"resp_68ee1b9a52c48190a71c3e7d6a0b2f5e","object":"response","status":"completed","usage":{"input_tokens":1830,"input_tokens_details":{"cached_tokens":1536},"output_tokens":196,"output_tokens_details":{"reasoning_tokens":128},"total_tokens":2026}}}
```

| Event | Meaning |
| --- | --- |
| `response.created`, `response.in_progress` | The response has been created / is being generated. The `status` in `response.created` may be `queued` |
| `response.output_item.added` / `.done` | An output item (message, function call, reasoning) starts / ends |
| `response.content_part.added` / `.done` | A content part within a message starts / ends |
| `response.output_text.delta` / `.done` | Text delta / full text |
| `response.reasoning_text.delta` / `.done` | Reasoning text delta / full reasoning text, returned by reasoning models |
| `response.function_call_arguments.delta` / `.done` | Function call argument delta / full arguments |
| `response.completed` | Normal end; `response.usage` is the usage for this call |
| `response.incomplete`, `response.failed` | The output was truncated or generation failed; these also carry the full `response` object |

- A Responses stream has **no** `data: [DONE]`; it ends with `response.completed` (or `response.incomplete`, `response.failed`). The gateway does not add `stream_options` to Responses requests, and usage appears only in the final event.
- If the upstream breaks off after output has started, the gateway appends a `response.failed` event at the end of the stream and ends it: `response.status` is `failed`, `response.error.code` is `server_error` and `response.id` is the ID of this stream's response, with no `response.completed`. Treat a stream that has no final event as failed; see [Timeouts and interrupted streams](https://platform.lyrouter.com/ai_gateway/docs/stream-errors/en).

**Streaming examples**

cURL:
```bash
curl -N https://api.lyrouter.com/v1/responses \
  -H "Authorization: Bearer $LYROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"MODEL_ID","input":"Write a two-line poem","stream":true}'
```

Python:
```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LYROUTER_API_KEY"],
    base_url="https://api.lyrouter.com/v1",
)
stream = client.responses.create(
    model="MODEL_ID",
    input="Write a two-line poem",
    stream=True,
)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
    elif event.type == "response.completed":
        print("\n", event.response.usage)
```

Node.js:
```js
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.LYROUTER_API_KEY,
  baseURL: "https://api.lyrouter.com/v1",
});
const stream = await client.responses.create({
  model: "MODEL_ID",
  input: "Write a two-line poem",
  stream: true,
});
for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  } else if (event.type === "response.completed") {
    console.log("\n", event.response.usage);
  }
}
```

For a comparison of streaming formats across protocols, see [Streaming](https://platform.lyrouter.com/ai_gateway/docs/streaming/en).

## Multi-turn conversations and function calls

There are two ways to hold a multi-turn conversation:

- **`previous_response_id`**: put the previous response's `id` in the next request and send only the new input; the upstream stores and assembles the context. This requires upstream support, and the previous turn must not have set `store: false`.
- **Carry the history yourself**: include the full history in `input` every time, appending the items from the previous `output` unchanged, followed by the new input. This does not depend on state stored by the upstream, so it is more reliable when the model has several channels or credentials. If you do not need the upstream to store anything, you can set `store: false`; in that case, to send reasoning items back, also set `"include": ["reasoning.encrypted_content"]`.

The gateway uses `previous_response_id` to group the requests of one continuation chain into a session, which you can see in the "Session" column of the [request logs](https://platform.lyrouter.com/ai_gateway/docs/logs/en). The gateway does **not** pin a continuation request to the upstream that created the previous response: when the model has several channels or credentials, a continuation request can land on an upstream that did not store that response, and the upstream returns an error saying the response cannot be found.

The full function call flow is as follows. The first request declares the tool:

```json
{
  "model": "MODEL_ID",
  "input": "What's the weather in Shanghai right now?",
  "tools": [
    {
      "type": "function",
      "name": "get_weather",
      "description": "Get the current weather for a city",
      "parameters": {
        "type": "object",
        "properties": { "city": { "type": "string" } },
        "required": ["city"],
        "additionalProperties": false
      },
      "strict": true
    }
  ]
}
```

When the model decides to call the tool, a `function_call` item appears in `output`, with `arguments` as a JSON string:

```json
{
  "id": "fc_68ee1c20a3f88190c2d4e6f8a0b1c3d5",
  "type": "function_call",
  "status": "completed",
  "call_id": "call_k3Jw9Xq2mZ7tVb1R",
  "name": "get_weather",
  "arguments": "{\"city\":\"Shanghai\"}"
}
```

Run the tool on the client, then return the result as `function_call_output`; `call_id` must match the call. The example below carries the history itself and includes the original `tools` again:

```json
{
  "model": "MODEL_ID",
  "input": [
    { "role": "user", "content": "What's the weather in Shanghai right now?" },
    {
      "type": "function_call",
      "call_id": "call_k3Jw9Xq2mZ7tVb1R",
      "name": "get_weather",
      "arguments": "{\"city\":\"Shanghai\"}"
    },
    {
      "type": "function_call_output",
      "call_id": "call_k3Jw9Xq2mZ7tVb1R",
      "output": "{\"temp_c\":22,\"condition\":\"cloudy\"}"
    }
  ],
  "tools": ["...same as in the first request..."]
}
```

The `tools` value in the example is abbreviated; a real request must include the full tool definitions. For a comparison with the Chat Completions format, see [Tool calls](https://platform.lyrouter.com/ai_gateway/docs/tool-call/en).

## Gateway behavior

- **No protocol conversion**: the gateway forwards requests to the upstream's `/responses` in the Responses protocol and never converts them to Chat Completions or Anthropic Messages, so both the model and the channel serving it must support Responses natively. When a model declares its supported endpoints and Responses is not among them, the gateway returns 400 `This model does not support the responses API` (`unsupported_model_api`) directly; models that declare none are forwarded as usual, and the upstream returns the error.
- **Requests are forwarded mostly as is**: the gateway makes only these changes: it replaces `model` with the selected channel's actual model name; with a virtual model, it merges parameters and the system prompt; it changes item IDs in `input` that start with `at_` to start with `msg_`, for compatibility with upstreams that require the `msg_` prefix; and it reorders the keys of objects in the request JSON (array order is unchanged) so identical prefixes reliably hit the upstream cache. `store`, `previous_response_id`, `background`, `include` and built-in tools are all forwarded as is.
- **Virtual models**: when `instructions` is empty, the virtual model's system prompt fills it; when it already has content, the prompt is appended after a blank line. Parameters are merged using Chat Completions field names, and `max_tokens` is not converted to `max_output_tokens`; see [Virtual models](https://platform.lyrouter.com/ai_gateway/docs/presets/en#when-the-system-prompt-is-injected).
- **Usage**: `usage` returns the upstream's values unchanged; the gateway does not fill in or rewrite them.
- **Fallback on failure**: when the upstream returns an error or the connection fails before output starts, the gateway switches channels as described in [Retries and fallback](https://platform.lyrouter.com/ai_gateway/docs/fallback/en), and if needed moves on to fallback models and the workspace's Default Model. The model that takes over must also support Responses, or the `unsupported_model_api` error above is returned. Once streaming output has started, the gateway no longer switches channels.
- **Codex CLI**: setting `wire_api = "responses"` in Codex calls this endpoint; the model and channel also need to support function tools. For setup, see [Codex](https://platform.lyrouter.com/ai_gateway/docs/codex/en).
- When quotas are configured, responses carry `x-aigw-quota-*` response headers; see [Gateway headers and extensions](https://platform.lyrouter.com/ai_gateway/docs/gateway-headers/en).

## Common errors

Errors returned by the gateway take the form `{"error":{"message":"...","type":"...","code":"..."}}`; only some errors include `code`. Errors passed through from the upstream unchanged may have a different shape: for example, when the response that `previous_response_id` points to does not exist, the result is 400 `{"request_id":"...","code":"InvalidParameter","message":"..."}`, with no outer `error`. Handle both when parsing.

| Status code | `error.message` | `error.code` | Cause and fix |
| --- | --- | --- | --- |
| 400 | `This model does not support the responses API` | `unsupported_model_api` | The model does not support Responses. Switch to a model that does, or use [Chat Completions](https://platform.lyrouter.com/ai_gateway/docs/chat-completions/en) instead; if this appears in Codex, switch models as well |
| 400 | `you must provide a model parameter` | `missing_model` | The request has no `model`, and the workspace has no Default Model either. Specify a model ID |
| 403 | `Model is not available in the model plaza` | `model_not_available` | The model is not enabled in the workspace; see [Unavailable models](https://platform.lyrouter.com/ai_gateway/docs/unavailable/en) |
| 503 | `all routed candidates exhausted` | `no_route_candidate` | No channel can serve this request, for example because `@vmodel/<name>` is misspelled and the virtual model does not exist. Check the name; see [Unavailable models](https://platform.lyrouter.com/ai_gateway/docs/unavailable/en) |
| Upstream status code | Upstream's original message | Upstream's, unchanged | For example, the response that `previous_response_id` points to does not exist on the current upstream. In the logs, "Status explanation" shows the source as "Upstream" |

- For 401 / 403 authentication issues, see [401 / 403: authentication and access](https://platform.lyrouter.com/ai_gateway/docs/auth-errors/en); for 429, see [429: rate limits and quotas](https://platform.lyrouter.com/ai_gateway/docs/rate-errors/en).
- 500 / 504 mean the connection to the upstream failed or the wait for the upstream timed out; the response body is an HTML page rather than JSON. See [Timeouts and interrupted streams](https://platform.lyrouter.com/ai_gateway/docs/stream-errors/en).
- For the full list, see [Errors and request identifiers](https://platform.lyrouter.com/ai_gateway/docs/errors/en). Gateway responses do not include a request ID. When troubleshooting, find the request in the [request logs](https://platform.lyrouter.com/ai_gateway/docs/logs/en) by API key and time, then copy the Request ID from the top of the log details.