LYRouter Docs

Protocol compatibility and limits

LYRouter offers three chat endpoints, Chat Completions, Anthropic Messages and Responses, while each provider channel uses a single upstream protocol: OpenAI-compatible or Anthropic. When the client endpoint and the channel protocol differ, the gateway converts requests, responses, streaming events and error formats automatically. The conversion keeps only fields that have a matching meaning on both sides; the rest are removed or rewritten, and these changes are not reported in response headers. Responses requests are never converted and can only be served by channels that provide a Responses endpoint. If your application relies on specific parameters, cache markers or usage fields, use this page to confirm the behavior for the combination you actually get.

Client endpoint × channel protocol

Endpoint the client calls OpenAI-compatible channel Anthropic-protocol channel
Chat Completions (/v1/chat/completions) Sent in its original format; the gateway only adjusts reasoning parameters and removes cache markers and reasoning_content from earlier messages (for the DeepSeek family, reasoning_content on current-turn assistant messages is kept) Converted into a Messages request; the response and streaming events are converted back to the Chat format
Anthropic Messages (/v1/messages) Converted into a Chat Completions request; the response and streaming events are converted back to the Anthropic format First converted into the internal Chat structure, then converted back into Messages and sent; when the platform turns on native passthrough, it can be sent as is, see below
Responses (/v1/responses) Sent as is to the channel's Responses endpoint; if the channel has no such endpoint, the upstream returns an error Not available. When the model declares its endpoint scope and Responses is not in it, the gateway returns 400 This model does not support the responses API directly
  • The gateway chooses the channel: the same model may have channels of both protocols, and a retry after a failure may move to a channel of the other protocol, so the same request can go through different conversions at different times. To confirm the format a particular request was actually sent in, open its log detail in Log analysis and check Provider, and Upstream request under Raw packets. Raw packet bodies are stored only after the workspace turns on content logging, except for failed requests.
  • Combinations without conversion are the most faithful: when a Chat Completions client goes through an OpenAI-compatible channel, or a Responses client goes through a channel that provides a Responses endpoint, the request is sent essentially as is. For applications with strict field requirements, ask the current Account Owner to keep only sources of the same protocol in the model details; see Connecting multiple providers.
  • Gateway rejections come first: when the model catalog declares the endpoint or input scope, requests that do not fit (such as a chat model calling the image endpoint, or an image sent to a text-only model) return 400 before any conversion; see Capabilities and compatibility.

Chat Completions → Anthropic-protocol channels

Request field Handling
Messages whose role is system or developer Merged into Anthropic's system, joined with blank lines. Only string content is accepted; system / developer messages whose content is an array are dropped
content of user and assistant messages Strings are converted as is; in content arrays only text and image_url are converted (data URLs become base64 images, http(s) URLs become URL images), and other types such as input_audio and file are dropped
tool messages, assistant.tool_calls Converted to tool_result blocks and tool_use blocks respectively; arguments is parsed into a JSON object, and becomes an empty object when parsing fails
assistant.reasoning_content Not forwarded
tools Converted to name, description and input_schema (taken from parameters)
tool_choice auto → auto, required → any, a specific function → tool; none is not forwarded, so when the request carries tools the model may still call a tool
max_tokens, max_completion_tokens Converted to max_tokens; 4096 is added when neither is present
stop Converted to stop_sequences; a string becomes a single-element array
metadata Only user_id is kept; other keys are dropped
enable_thinking, thinking_budget Converted to thinking: {"type": "enabled", "budget_tokens": N} when there is a budget, {"type": "adaptive"} when thinking is only turned on without a budget, and not sent when thinking is off
n, logprobs, top_logprobs, response_format, frequency_penalty, presence_penalty, seed, parallel_tool_calls, logit_bias, functions, function_call, reasoning_effort, stream_options Dropped
Other fields (temperature, top_p, top_k, stream, service_tier and so on) Sent as is; the upstream decides whether to accept fields Anthropic does not recognize

The response is converted back to the Chat format: text blocks are merged into message.content, tool_use blocks become tool_calls, and thinking content goes into reasoning_content; stop_reason is mapped to finish_reason (end_turn, stop_sequence, pause_turn and refusal → stop, max_tokens → length, tool_use → tool_calls). Because n is dropped, choices always has a single item.

Thinking is turned off when a tool call is forced: when tool_choice is required or a specific function and the request sets no reasoning parameters, the gateway turns thinking off for you, so that the upstream does not reject the request with "thinking mode does not support forced tool calls". This rule applies to requests from any endpoint sent to Anthropic-protocol channels or to channels that use enable_thinking, such as Alibaba Cloud Model Studio (Bailian).

Anthropic Messages → channels

/v1/messages requests are first converted into the internal Chat structure, then sent according to the channel protocol. After this step:

Request content Handling
system (a string or an array of text blocks) Merged into one piece of text, with blocks joined by blank lines; when several blocks carry cache_control, only the last one is kept
Blocks in messages text, image, tool_use, tool_result, thinking and redacted_thinking are converted; document and other block types are dropped
A user message that contains both tool_result and text or images Only tool_result is kept. Put any text that should accompany it in the next user message
tools Converted by name, description and input_schema. Server tools that have only type and name (such as web_search) become ordinary function tools with the same name, which the client has to execute itself
tool_choice auto, any and tool are converted; none and disable_parallel_tool_use are not forwarded
max_tokens, stop_sequences, stream, temperature, top_p, top_k, service_tier Kept
thinking {"type": "disabled"} counts as off and any other type as on, with budget_tokens kept as the thinking budget; for Anthropic-protocol channels it is restored to enabled with a budget or adaptive without one
metadata, context_management, output_config Restored for Anthropic-protocol channels (metadata keeps only user_id); removed for OpenAI-compatible channels
Other top-level fields Not forwarded

For OpenAI-compatible channels, the thinking switch is passed in the provider's own form: channels such as Alibaba Cloud Model Studio (Bailian) receive enable_thinking / thinking_budget; the official OpenAI and DeepSeek channels do not support these two fields, so they are removed; providers the gateway has not adapted receive the two fields as is, and the upstream decides whether they take effect. The effort setting in output_config is not converted to reasoning_effort. Thinking blocks in earlier messages and all cache_control markers are removed as well.

The response is converted back to the Anthropic format:

  • stop_reason is only ever end_turn, max_tokens or tool_use; the upstream's stop_sequence, pause_turn and refusal all become end_turn, and the stop_sequence field does not appear in the response (streaming and non-streaming alike). This also applies to Anthropic-protocol channels.
  • Thinking content (reasoning_content) returned by OpenAI-compatible channels becomes thinking events in streaming responses, without a signature; non-streaming responses do not include thinking content.
  • When the answer contains only thinking content (for example, it was cut off by max_tokens during the thinking phase), content in a non-streaming response is currently an empty object {} rather than an empty array []. Check its type before iterating over content as an array.
  • For Anthropic-protocol channels, thinking block signatures make the round trip unchanged and can be sent back in the next turn. When earlier messages contain thinking blocks without a signature (for example, the previous turn was answered by an OpenAI-compatible channel), this turn drops all earlier thinking blocks and does not turn thinking on, so that the upstream does not reject the request for failing signature verification.

Reasoning parameters

The three reasoning parameters belong to different providers. The gateway does not convert between them; it only removes the ones the selected channel does not recognize. Except for the Anthropic-protocol row, each row in the table below refers to OpenAI-compatible channels:

Channel reasoning_effort enable_thinking thinking_budget
OpenAI Kept Removed Removed
Anthropic protocol Removed Converted to thinking Converted to thinking
Alibaba Cloud Model Studio (Bailian, including Qwen) Removed Kept Kept
DeepSeek official Removed Removed Removed (the model decides on thinking)
Other providers Sent as is Sent as is Sent as is

Different channels of the same model can fall into different rows. When you need stable reasoning behavior, verify each channel, or fix the reasoning parameters with a virtual model.

Cache markers cache_control

Combination Handling
/v1/messages → Anthropic-protocol channel Markers on system, text blocks and tool definitions are kept; markers on several system blocks are merged into one; markers on image blocks or tool_result blocks are not forwarded
/v1/chat/completions → Anthropic-protocol channel Markers on the text and image parts of content arrays and on each tool in tools are kept; for system messages, put the marker on the message object (at the same level as role and content) and use a string for content
Any endpoint → OpenAI-compatible channel All removed, including for providers that support explicit caching, such as Alibaba Cloud Model Studio (Bailian). These channels rely on the upstream's automatic prefix caching
  • 1-hour cache: when a marker contains "ttl": "1h", the gateway sets the anthropic-beta request header to extended-cache-ttl-2025-04-11, replacing any value the client had in that header.
  • Virtual model cache setting: when a virtual model has caching turned on and the request itself carries no markers, the gateway adds a marker to the system prompt only for requests sent to Anthropic-protocol channels; when the request carries its own markers, the request wins.
  • Whether markers take effect and how much they hit is shown under Explicit cache write and Input (cache hit) in the logs; optimization tips are in Reducing costs and improving cache reuse.

The usage field

The usage in a response is counted differently depending on the combination:

Client endpoint ← channel usage in the response
Chat Completions ← OpenAI-compatible Returned as is from the upstream: prompt_tokens includes the cache hits, and the hit count is in prompt_tokens_details.cached_tokens (prompt_cache_hit_tokens for DeepSeek)
Chat Completions ← Anthropic protocol prompt_tokens equals the upstream's input_tokens and excludes cache hits and cache writes; non-streaming responses also carry cache_read_input_tokens and cache_creation_input_tokens, while the last streaming chunk has only prompt_tokens, completion_tokens and total_tokens
Anthropic Messages ← Anthropic protocol input_tokens excludes the cache; cache_read_input_tokens and cache_creation_input_tokens appear only when greater than 0
Anthropic Messages ← OpenAI-compatible input_tokens = upstream prompt_tokens − cache hits, cache_read_input_tokens = cache hits, output_tokens = upstream completion_tokens
Responses Returned as is from the upstream: input_tokens includes the cache hits, and the hit count is in input_tokens_details.cached_tokens
  • In streaming Chat Completions, the gateway always requests usage, and the last chunk carries the usage; Responses and Anthropic Messages report usage in their end events according to their own protocols.
  • Request logs are unified: the four items Input (cache miss), Input (cache hit), Output and Explicit cache write in the logs are calculated the same way for every protocol combination and never overlap. When you reconcile costs or calculate hit rates, go by the logs and usage analytics; do not add up prompt_tokens from different combinations directly.

Error formats and stream end markers

Client endpoint Error response body Normal end of a stream Upstream breaks off after streaming has started
Chat Completions {"error": {"message", "type", "code"}}; code appears only on some errors data: [DONE] A data: {"error": {..., "code": "stream_interrupted"}} is appended, with no data: [DONE]
Anthropic Messages {"type": "error", "error": {"type", "message"}}, with no code event: message_stop An event: error is appended, with no message_stop
Responses Same as Chat Completions; upstream errors are forwarded as is A response.completed event, with no [DONE] A response.failed event (error.code server_error) is appended, with no response.completed
  • On /v1/messages, both gateway errors and upstream errors are converted to the Anthropic format, with message unchanged and error.type determined by the status code (such as 401 → authentication_error, 429 → rate_limit_error, 503 → overloaded_error).
  • A 500 / 504 from a failed upstream connection or a wait timeout returns an HTML error page, not JSON, on every endpoint; see Timeouts and interrupted streams.
  • Parameters that chat endpoints drop during conversion are not written to the X-Gateway-Unsupported-Parameters response header; this response header and the X-Gateway-Strict-Parameters request header currently apply only to image and video endpoints; see Gateway headers and extensions.

All error codes are listed in Errors and request identifiers.

Server-side state in Responses

Responses requests are forwarded as is, and store, background and previous_response_id are all handled by the upstream. Response objects are stored at the upstream that created them:

Operation What the gateway does Limits
A new request with previous_response_id Uses it to group the request into the same session (the Session column in the logs), then selects a channel as usual The request is not guaranteed to reach the same upstream. When the channel or credential changes, the new upstream cannot find the previous response and returns its own error
  • Multi-turn conversations: the Lowest Price and Load balanced strategies keep requests from the same API key to the same model on one channel for 30 minutes, so most follow-up requests go back to the original upstream; but a retry after a failure, a strategy change or the end of the binding period can still move them to another channel; see Channels and routing. For reliable multi-turn conversations, keep the history on the client and send the full context in input every turn.
  • Virtual model parameters: when you call a virtual model through Responses, its Inference params are merged into the request under their Chat Completions field names; for example, max_tokens is not changed to max_output_tokens. The upstream returns 400 when it does not accept these fields, so for virtual models used by Responses clients, set only parameters that have the same name in both endpoints (such as temperature and top_p).

Native passthrough

Native passthrough lets /v1/messages requests skip the conversion above when they are sent to Anthropic-protocol channels: apart from model being replaced with the upstream model name, the request body is sent exactly as the client sent it, and the response is returned as the bytes the upstream returned; billing and logging are not affected. The platform can turn on passthrough for responses only, or for both requests and responses; both are off by default, and every /v1/messages request goes through conversion.

A client that needs to be sure its request is not rewritten can add a request header that requires passthrough; when passthrough is not possible, the gateway rejects the request instead of converting it silently:

curl https://api.lyrouter.com/v1/messages \
  -H "x-api-key: $LYROUTER_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "x-gateway-passthrough: require" \
  -H "Content-Type: application/json" \
  -d '{"model": "MODEL_ID", "max_tokens": 1024, "messages": [{"role": "user", "content": "Hello"}]}'
x-gateway-passthrough Behavior
Absent, or prefer Passthrough when possible; otherwise falls back to conversion automatically, and the request completes as usual
require (case-insensitive) Passthrough is mandatory; when it is not possible, returns 400 native passthrough was required by `x-gateway-passthrough: require` but is not available for this request (<reason>)

The reason in parentheses tells you which condition was not met:

Reason Meaning Fix
route_disabled The platform has not turned on sending requests as is for this endpoint (the default) Remove this header and accept conversion
client_protocol The request was not sent to /v1/messages This header only works with Anthropic Messages
wire_protocol The selected channel does not use the Anthropic protocol Switch to a model served by Anthropic-protocol channels
endpoint_type Not a chat request Remove this header
channel_disabled The platform has turned off passthrough for this channel Switch to another model or remove the header
preset_injection The virtual model sets a system prompt, reasoning parameters, locked parameters or caching, which requires rewriting the request Call the model ID directly, or remove these settings from the virtual model
shape_oai The request body contains OpenAI-only fields (such as response_format or n), message roles other than user / assistant, or tools in the OpenAI format Rewrite the request in pure Anthropic format
decision_error The gateway hit an error while deciding Retry later; contact support if it persists

Passthrough applies only to the channel the request is actually sent to; if a retry after a failure moves to an OpenAI-compatible channel, a request with require also returns 400.

FAQ

I set response_format. Why is the response not JSON?

Answer: When the request is routed to an Anthropic-protocol channel, response_format is dropped during conversion. Check Provider for this request in the logs; when you need stable JSON output, state the format in the prompt, or keep only OpenAI-compatible sources; see Capabilities and compatibility.

When I call Claude models from Claude Code, are requests rewritten?

Answer: They go through one conversion, but thinking block signatures, cache markers, metadata, context_management and output_config are all restored. What to watch out for are document blocks, text in the same message as a tool_result, server tools and the finer stop_reason values; see "Anthropic Messages → channels" above. Confirm cache hits under Input (cache hit) in the logs.

One application uses both the OpenAI SDK and the Anthropic SDK. How do I get unified usage figures?

Answer: Do not add up the usage in responses directly; the two formats count the cache differently. Go by the request logs and Usage analytics, which already put the four token types on the same basis; see Troubleshooting with logs and Usage attribution and analytics.