Rerank documents

Scores a set of documents by relevance to a query and sorts them, for refining retrieved results in retrieval-augmented generation (RAG) and semantic search; to convert text into vectors, use Create embeddings. OpenAI has no rerank endpoint: LYRouter sends the request body unchanged to the channel's /rerank address, replacing only model, so field names, values and response formats follow the rerank documentation of the provider the model belongs to. Requests and responses are JSON, and streaming is not supported; the OpenAI SDK has no rerank method, so call this endpoint over HTTP directly.

Authentication

Send your API key in a request header:

Authorization: Bearer <API Key>

Bearer is case-sensitive. You can use x-api-key: <API Key> instead; when both are present, only Authorization is read. Create API keys on the workspace's API Keys page. Your API key is used only to authenticate with LYRouter and is never forwarded to the upstream provider. See Base URLs and authentication; to set the LYROUTER_API_KEY environment variable, see Your first model call.

Headers

Request body

The table below lists fields common to popular rerank APIs such as Jina and Cohere; actual field names and values follow the rerank documentation of the provider the model belongs to.

Other fields are forwarded unchanged to the upstream; whether they take effect depends on the model.

Choosing a model

The rerank models you can use are those in your workspace's model marketplace: search the model marketplace for keywords such as rerank, then copy the model ID to replace MODEL_ID in the examples.

You can also filter with GET /v1/models. Each model returned has an api_types field; a model whose list includes rerank can call this endpoint:

curl -s https://api.lyrouter.com/v1/models \
  -H "Authorization: Bearer $LYROUTER_API_KEY" \
  | jq -r '.data[] | select((.api_types // []) | index("rerank")) | .id'

Models that do not declare api_types do not appear in the filtered result; confirm them in the model marketplace. For field descriptions, see List models; to filter embedding models, see Create embeddings.

Request examples

The Python example uses httpx, which is installed along with openai.

curl https://api.lyrouter.com/v1/rerank \
  -H "Authorization: Bearer $LYROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "query": "How do I see which model a call actually used?",
    "documents": [
      "Model (actual) in the log details shows the model that gave the final answer.",
      "API keys are created on the workspace API Keys page.",
      "The Default Model can be set on the Smart routing page."
    ],
    "top_n": 2,
    "return_documents": true
  }'

import os
import httpx

resp = httpx.post(
    "https://api.lyrouter.com/v1/rerank",
    headers={"Authorization": f"Bearer {os.environ['LYROUTER_API_KEY']}"},
    json={
        "model": "MODEL_ID",
        "query": "How do I see which model a call actually used?",
        "documents": [
            "Model (actual) in the log details shows the model that gave the final answer.",
            "API keys are created on the workspace API Keys page.",
            "The Default Model can be set on the Smart routing page.",
        ],
        "top_n": 2,
        "return_documents": True,
    },
    timeout=60,
)
resp.raise_for_status()
for item in resp.json()["results"]:
    print(item["index"], item["relevance_score"])

const resp = await fetch("https://api.lyrouter.com/v1/rerank", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LYROUTER_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "MODEL_ID",
    query: "How do I see which model a call actually used?",
    documents: [
      "Model (actual) in the log details shows the model that gave the final answer.",
      "API keys are created on the workspace API Keys page.",
      "The Default Model can be set on the Smart routing page.",
    ],
    top_n: 2,
    return_documents: true,
  }),
});
if (!resp.ok) throw new Error(await resp.text());
const { results } = await resp.json();
for (const item of results) console.log(item.index, item.relevance_score);

Response

A successful response comes from the provider and the gateway does not rewrite it; an error returns an error object. The 200 example below is a common shape; the 400 means the request landed on a model that does not support the rerank endpoint (usually the Default Model; see "Gateway behavior" below); the 403 means the model is not enabled in this workspace and the workspace has no Default Model.

{
  "model": "MODEL_ID",
  "results": [
    { "index": 0, "relevance_score": 0.92, "document": { "text": "Model (actual) in the log details shows the model that gave the final answer." } },
    { "index": 2, "relevance_score": 0.31, "document": { "text": "The Default Model can be set on the Smart routing page." } }
  ],
  "usage": { "total_tokens": 86 }
}

{
  "error": {
    "message": "This model does not support the rerank API",
    "type": "invalid_request_error",
    "code": "unsupported_model_api"
  }
}

{
  "error": {
    "message": "Model is not available in the model plaza",
    "type": "access_denied",
    "code": "model_not_available"
  }
}

Billing

Billed by the billing items listed in the model marketplace, with usage taken from usage in the upstream response; models priced per call read search_units from it. Unit prices are those shown in the model marketplace, and the actual cost of each call is in the logs. For how to estimate costs, see Estimating costs and cache pricing.

Gateway behavior

• Requests forwarded unchanged: the gateway only replaces model with the channel's real model name and forwards the other fields unchanged (for calls through a virtual model, see below). The gateway re-encodes the JSON: object keys are sorted into a fixed order, and numbers keep at most 14 significant digits.
• Channel fallback: when a model has several channels, the gateway tries them in turn according to the workspace's routing strategy, moving to the next channel when one fails.
• Default Model fallback: when the workspace has a Default Model set in Smart routing, the three cases below land on the Default Model. The Default Model is usually a chat model, so you get an error instead of the result you expected (when the Default Model declares api_types, the gateway returns 400 This model does not support the rerank API; when it does not, the request is forwarded to the upstream and the upstream returns the error):
  • the request has no model;
  • model is misspelled, or the model is not enabled in this workspace;
  • every channel of the model failed. In this case the upstream's real error is replaced by the Default Model's error; check the reason for each earlier attempt under "Upstream call attempts" in the log details.
• Virtual models and Auto: virtual models can be used with this endpoint, provided that every model in the virtual model's model list is a rerank model. A virtual model's system prompt is not injected, but its inference params are merged into the request body by field name, so do not set inference params on virtual models used for rerank. The candidates of lyrouter/auto are chat models; do not use it with this endpoint.
• Guardrails: bans, Model access and Client access in guardrails apply as usual; Sensitive data, Prompt injection and Intent security detection do not apply to this endpoint.

Common errors

Error responses take the form {"error":{"message":"...","type":"...","code":"..."}}; only some errors include code.

• Errors such as too many or too long documents are returned by the upstream provider; the status code and error text come from the upstream.
• When you call this endpoint with a model that does not declare api_types (see List models), the gateway does not return 400 unsupported_model_api; it forwards the request to the upstream, which returns the error.
• The gateway resolves the model before it checks Content-Type: when model is missing or the model is not enabled, a model-related error comes first (400 missing_model, 403 model_not_available); only with a valid model do you get 400 unsupported content-type: ….
• For 401 / 403 authentication issues, see 401 / 403: authentication and access; for 429, see 429: rate limits and quotas. For the full list, see Errors and request identifiers.