Skip to main content
This guide is for developers sending traffic. If you already call OpenAI or Anthropic, you keep your code and change two things: the base URL and the API key.
You’ll need the gateway base URL (such as https://your-gateway.example.com/v1) and an API key (it starts with sk-olga-). Your operator creates these in Set up the gateway.

Send a request

Point your client at the gateway and use your gateway key. Everything else about your code stays the same.
The gateway speaks the OpenAI Responses API at /v1/responses and the OpenAI Embeddings API at /v1/embeddings, plus the Anthropic Messages API at /v1/messages. Use whichever your SDK targets. Streaming on the text-generation endpoints behaves exactly as it does against the provider directly: set stream=True (or stream: true) and read the events as usual.

What you just got

On its way to the provider and back, your request was also:
  • Checked against your usage limits, and refused if it would exceed one.
  • Screened by guardrails for PII and prompt injection, on supported content endpoints.
  • Traced to your Openlayer project, with model, tokens, latency, and cost.
None of that needed a line of code in your app. Embedding requests skip content guardrails so redaction can’t change the vectors. Their traces show an embedding step and its count and dimensions, without dumping the vector values. Every gateway request becomes a trace in Openlayer

If something goes wrong

The most common errors are below. Errors covers every status code, the exact messages, and which ones are worth retrying.
The key is missing, mistyped, or has been disabled or deleted. Confirm you’re sending the sk-olga- key your operator gave you, as Authorization: Bearer <key>.
A usage limit on your key or group has been reached. Ask your operator to review your usage limits.
A guardrail blocked the request or the response. The message is the guardrail’s reason, such as “Detected high-risk PII entities: US_SSN”. See Guardrails.
The message tells you which of these causes you hit:
  • A guardrail is still loading, for example “The pii guardrail is still loading on this replica”. This is temporary and happens shortly after a guardrail is added or changed, or after the gateway starts. The response carries a Retry-After header; retry after a few seconds. Most SDKs’ built-in retries handle this for you.
  • The provider isn’t connected, or its API key isn’t set on the gateway host. Retrying won’t help. Your operator can fix this in Connect providers.
  • The endpoint can’t be routed to the selected provider or rule. Embeddings and image generation require an OpenAI-format provider, and embeddings can’t use a model-based routing rule. Your operator can fix this in Route requests.