> ## Documentation Index > Fetch the complete documentation index at: https://docs.slng.ai/llms.txt > Use this file to discover all available pages before exploring further. # Integration guide > Call the Context Router from any OpenAI-compatible client: authentication, requests, streaming, caching, per-call variables, and BYOK configuration. The Context Router is a single endpoint that speaks the OpenAI Chat Completions protocol. Point an OpenAI-compatible client at it, authenticate with your SLNG API key, and every turn gets routing, caching, and in-region execution without changing how you call the model. ## Endpoint and authentication The router exposes two OpenAI-compatible surfaces. Both share the same auth, routing, caching, and features. ```text Chat Completions theme={null} POST /chat/completions Authorization: Bearer Content-Type: application/json ``` ```text Responses theme={null} POST /responses Authorization: Bearer Content-Type: application/json ``` The `/responses` endpoint is a translation layer over the **same** router: identical auth, routing, caching, and features. Everything in this guide applies to both surfaces unless a section says otherwise. Authenticate with **only** your `SLNG_API_KEY`, passed as a Bearer token. The OpenAI SDK's `api_key` parameter sets this header for you, so you never set internal auth headers yourself. ```python theme={null} from openai import OpenAI client = OpenAI( # Region-specific; use the exact base URL SLNG gave you. base_url="https://in.context-router.slng.ai/v1", api_key="YOUR_SLNG_API_KEY", ) ``` ## Making a request A request is a standard OpenAI Chat Completions call. ```python theme={null} resp = client.chat.completions.create( model="slng/auto", messages=[ {"role": "system", "content": "You are a helpful clinic assistant."}, {"role": "user", "content": "What are your opening hours?"}, ], ) print(resp.choices[0].message.content) ``` ### Streaming Streaming works exactly as in OpenAI, for **every** model the router serves (reasoning and non-reasoning alike). **Always set** `stream_options.include_usage=True` so you receive the final usage trailer chunk. ```python theme={null} stream = client.chat.completions.create( model="slng/auto", messages=[...], stream=True, stream_options={"include_usage": True}, ) ``` What to expect on the stream: * **Reasoning blocks and control tokens are removed for you.** If your model emits a visible reasoning block (``, ``, ``, ``) or a stray control token (``, `<|im_end|>` and similar), the router strips it before the chunk reaches you, so your TTS never reads it aloud. Setting `slng_pure_proxy` suspends this. * **All answer content arrives before the** `finish_reason: "stop"` **chunk**, so stopping there is safe. * **The `usage` block rides the final chunk.** That is why `include_usage` matters. * **The stream always ends cleanly.** If the model fails mid-stream, the router still sends a final `finish_reason: "stop"` chunk and `data: [DONE]`, so your client is never left hanging. **Ignore unknown fields in your client.** Nearly every client already does, including the official OpenAI SDKs. If yours uses strict deserialization (`deny_unknown_fields` in serde, `DisallowUnknownFields` in Go, or a hand-written parser that rejects extras), turn it off for this endpoint. The router forwards a provider's own extra fields verbatim, and some are required on the next turn. On a stream a strict decoder fails quietly: it skips the chunks it cannot parse, so you get a truncated answer with a `200`. ## Available parameters | Field | What it does | | :----------------------------- | :------------------------------------------------------------------------------------------------------------------------- | | `model` | `slng/auto`, the default when omitted, or the name of one entry in your configuration to pin it. | | `messages` | Standard OpenAI messages array. `developer` is treated like `system`. | | Sampling params | `temperature`, `top_p`, `max_tokens`, `max_completion_tokens`, and so on. | | `stream` | `true` or `false`. | | `stream_options.include_usage` | Set `true` to get the usage trailer. | | `tools` / `tool_choice` | Standard tool-calling, fully supported. An answer containing tool calls is never cached. | | `template_variables` | Per-call personalization values for `{{name}}` placeholders. See [Per-call values](#per-call-values-template_variables). | | `slng_config` | Your model configuration inline: endpoints and provider keys. See [BYOK Context Router](#byok-context-router-slng_config). | | `slng_agent_id` | Required, unless you send the `X-Slng-Agent-Id` header instead. | | `slng_session_id` | Required, unless you send the `X-Slng-Session-Id` header instead. | | `slng_analytics` | Default `false`. Keeps a copy of the request and answer for your analytics. See [Shadow mode](#shadow-mode). | | `slng_pure_proxy` | Default `false`. Returns your model's answer untouched. See [Shadow mode](#shadow-mode). | ### `slng/auto` vs a named model Recommended. SLNG picks from your configuration: the preferred tier first, splitting traffic by weight, with automatic retry on failure. The exact name of one entry in your configuration. It pins the request to that entry, so there is no failover to another one, and its traffic keeps its own cache entries. ## Required fields Your org identity is managed by SLNG. The agent ID and session ID are yours to send, and both are **required on every request**. * **Agent ID** names the logical agent a call belongs to, and scopes its cache. Use a stable value per agent. When you change that agent's prompt in a meaningful way, give it a fresh ID, for example `clinic-scheduler-v2`, so answers produced under the old prompt are not reused. * **Session ID** identifies a single call. Use a value unique to each call, like a UUID, and keep it the same across all turns of that call. Send each one as a header (`X-Slng-Agent-Id`, `X-Slng-Session-Id`) or as a top-level body field (`slng_agent_id`, `slng_session_id`). The body field wins if you send both. Values are at most 256 characters and cannot contain spaces, commas, pipes (`|`) or braces (`{` `}`). A request missing either ID, or carrying a malformed one, returns a `400` with code `missing_client_id` or `invalid_client_id`. ## Reading the response Every request returns a set of `x-slng-*` headers that tell you how the answer was produced. | Header | Present on | Meaning | | :----------------------- | :-------------------------------------- | :------------------------------------------- | | `x-slng-request-id` | Every response (including errors). | Quote it in a support request. | | `x-slng-response-source` | Every successful (2xx) chat completion. | `llm` or `cache`. | | `x-slng-cache-layer` | Cache hits only. | Which cache layer matched. | | `x-slng-model` | Live LLM answers only. | The model that answered, after any failover. | On an error, only `x-slng-request-id` comes back, because no answer was produced. Two outcomes are possible on a successful request: `x-slng-response-source: llm`. The request was routed to the LLM, and `usage` carries the tokens you were billed for. `x-slng-response-source: cache`. No LLM ran. `usage` carries the original answer's tokens, so it is what the cache saved rather than what this call cost. ## Response caching The Context Router is a PII-aware service that can serve a repeated turn straight from a cache instead of calling the LLM again. Caching is on by default, and a turn that qualifies is served from cache automatically. Cache layers are checked fastest-first; the first hit wins. | Layer | Match style | | :------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------- | | `l1_full_input`, `l2_full_input` | Whole-request match. | | `l1_exact`, `l2_exact` | Exact match. | | `l2_norm_exact` | Normalized-exact match. | | `l2_canonical` | Canonical-class match. | | `prewarm_full_input`, `prewarm_exact`, `prewarm_norm_exact`, `prewarm_canonical` | The same match styles, over answers SLNG pre-loaded for your agent ahead of time. | Pre-warmed answers are set up together with you, per agent. ### What SLNG does not store So one caller's answer is never replayed to another, some responses are not stored. The caller still gets their answer; only the caching is skipped. * **Any reply containing a number, in any script.** A number is usually specific to one caller or record, so identifiers, prices and dates are never cached. * **Personal information**, such as names, addresses and emails. * **Any reply carrying a placeholder the router did not fill in itself.** ## Per-call values (`template_variables`) Voice agents need per-call values inside the system prompt: the caller's name, their city, an appointment time. Instead of building those strings yourself before every request, write a placeholder once and let SLNG fill it in. Do this rather than building the string yourself. The stored answer then holds the placeholder instead of the caller's data, which is what makes a personalized answer cacheable and lets your callers share one entry. The model sees the same final prompt either way. ```python Do theme={null} # Keep placeholders in the prompt, send the values separately. {"role": "system", "content": "You are calling {{caller_name}} in {{city}}."} # ...with, in the same request: # "template_variables": {"caller_name": "Rajesh", "city": "Mumbai"} ``` ```python Don't theme={null} # Build the prompt string yourself per call. {"role": "system", "content": f"You are calling {caller_name} in {city}."} ``` The reverse case matters just as much. A value that steers the answer is not a template variable, however much it varies per call. ```python Do theme={null} # Write the steering value into the prompt text. {"role": "system", "content": "Reply to the caller in Spanish."} ``` ```python Don't theme={null} # The reply language decides what the answer IS, not what it says. {"role": "system", "content": "Reply to the caller in {{session_language}}."} # "template_variables": {"session_language": "Spanish"} # <-- wrong field ``` **Syntax.** Use double braces around the variable name: `{{caller_name}}`, `{{city}}`. Names are letters, digits, and underscore. Single braces (`{name}`), Jinja, and `${var}` are **not** placeholders. Only `{{name}}` is. Substitution applies to your prompt messages, the `system` and `developer` roles. Your `user` and `assistant` message content is never modified, so a literal `{{...}}` a caller happens to type is left exactly as-is. Here is a full request using template variables: ```python theme={null} resp = client.chat.completions.create( model="slng/auto", messages=[ {"role": "system", "content": "You are calling {{caller_name}} in {{city}}."}, {"role": "user", "content": "hello?"}, ], extra_body={ "template_variables": {"caller_name": "Rajesh", "city": "Mumbai"}, }, ) ``` The model receives `You are calling Rajesh in Mumbai.` **Missing values are rejected.** If your request references a `{{name}}` you did not supply in `template_variables`, the router returns a `422`. ```json theme={null} { "error": { "message": "Missing required template variables: appointment_time, city", "type": "invalid_request_error", "param": "template_variables", "code": "missing_template_variables" } } ``` ### What belongs in `template_variables` * **Do** carry personalization values that get spoken or echoed in the conversation: the customer's name, the agent's name, the company name, an appointment detail rendered as text. * **Don't** carry values that steer or change the content of the answer itself: the response language, a plan or tier that changes which policy is described, a region that changes which rules apply. * The language case deserves spelling out. A language variable is fine when it is just text to be said, for example the agent confirming "So your preferred language is `{{language}}`, correct?" during the call. It is not fine when it controls the language the model answers in: two callers who chose different languages would share one cached answer, and one would hear the wrong language. **Limits.** Up to 64 variables per request, names up to 64 characters, values up to 4000 characters. Exceeding these returns a `422`. **Runtime variables.** To leave a placeholder deliberately unfilled, because your own orchestrator fills it mid-call, set `template_vars_strict: false` on your configuration. The placeholder then passes through untouched, and an answer that echoes one is never cached. ## BYOK Context Router (`slng_config`) There are two ways the router can know which model(s) should answer your requests: You send the configuration on each request. It runs against exactly what you sent, and you change models by changing your own code. Register a BYOK LLM key in the dashboard and the router routes `slng/auto` to it with nothing extra in the request. Richer setups, such as several tiers or failover groups, SLNG configures with you. A request carrying `slng_config` ignores the stored configuration entirely, so anything you rely on has to be in the object you send. It holds your own endpoints and provider keys, so treat it like a credential. ### The configuration shape A configuration is a set of numbered **tiers**, tried in order of preference: `"1"`, `"2"`, `"3"`, three at most. Within a tier, traffic splits by `weight`. If the chosen model fails to answer (a `5xx`, a timeout, or a `429`), the router retries **once** against the next option. With a single entry it retries once against that same endpoint when the failure was transient. Each entry has: | Field | What it is | | :------------------ | :----------------------------------------------------------------------------------------------------------------------------------- | | `model` | The name you use to refer to this entry. | | `weight` | This entry's share of the tier's traffic (1 to 100; a tier's weights sum to 100). | | `endpoint` | Where the model is served, including your credentials for it. | | `endpoint.model_id` | The model name to send to the provider, when it differs from your `model` label. Optional except on `bedrock`, where it is required. | The smallest useful configuration is one tier with one OpenAI-compatible endpoint: ```python theme={null} resp = client.chat.completions.create( model="slng/auto", messages=[{"role": "user", "content": "Hello"}], extra_body={ "slng_config": { "tiers": { "1": [ { "model": "gpt-4o-mini", "weight": 100, "endpoint": { "url": "https://api.openai.com/v1", "api_key": "sk-...", # YOUR provider key, not your SLNG key }, } ] } }, }, ) ``` Your SLNG key stays in the `Authorization` header as always. The `api_key` inside the endpoint is your own key **for that provider** (here, OpenAI). It is used to make that request's model call and is never echoed back in any error message. ### Supported providers The `endpoint` object defaults to `provider: "openai-compat"`, which covers any service that speaks the OpenAI Chat Completions API (OpenAI itself, Azure-hosted alternatives, Groq, self-hosted vLLM, and so on). Four more providers are supported; each needs its own fields on the `endpoint` object. | `provider` | Required endpoint fields | | :------------------------ | :--------------------------------------------------------------------- | | `openai-compat` (default) | `url`, `api_key` | | `openai-responses` | `url`, `api_key` | | `azure` | `url`, `api_key`, `azure_deployment`, `api_version` | | `vertex` | `vertex_credentials`, `vertex_location` | | `bedrock` | `aws_access_key_id`, `aws_secret_access_key`, `aws_region`, `model_id` | On `azure`, `url` is the resource root, not the full deployment URL. On `openai-compat`, add `auth_header` when the provider wants its key in a custom header. Fields belonging to another provider are rejected rather than ignored. For example, an Azure OpenAI entry looks like: ```json theme={null} { "model": "my-azure-gpt4o", "weight": 100, "endpoint": { "provider": "azure", "url": "https://my-resource.cognitiveservices.azure.com/", "api_key": "...", "azure_deployment": "gpt-4o-deploy", "api_version": "2024-12-01-preview" } } ``` ### Limits and errors * The `slng_config` object must stay under **256 KB** when serialized; larger returns a `400`. * A configuration that fails validation returns a `400` starting with `invalid slng_config:` and a description of the problem. Your credentials never appear in these messages. * Sending `slng_config` as anything other than a JSON object (a string, a list) returns a `422` with code `invalid_slng_config`. ## Shadow mode Two optional flags let you route production traffic through SLNG and measure what it would save, before you let it change anything. Both default to `false` and need no setup. | Flag | What it does | | :---------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `slng_analytics` | Keeps a copy of the request and the answer for your analytics, in addition to serving the call normally. It changes nothing about the answer, your latency, which model answers, or caching. Copies stay in your region. | | `slng_pure_proxy` | Returns your own model's answer untouched: no cached answers, no prepared replies, no changes to the text. Your configuration still applies, including failover. | Send both together and you have a shadow trial. You receive exactly what your own model produces, while the router measures what it could have served from cache and what that would have saved. Going live is removing the two flags: same URL, same key, same request body. A trial measures; it does not speed anything up. A turn the router would have answered from cache goes to your model instead, so trial latency reflects your own model. Send your prompt in template form with `template_variables` for the measurement to be meaningful.