Skip to main content
The Openlayer Gateway is a single endpoint that sits between your apps and your LLM providers and enforces your rules on every request, in real time. It blocks calls that go over budget, stops prompt injection, and redacts PII before a request reaches a provider or a response reaches a user. Every request that flows through the gateway is also traced to your Openlayer project, so the monitoring you already rely on keeps working.

​
How it works

1

Point your app at the gateway

Keep your existing OpenAI- or Anthropic-compatible client. Point its base URL at the gateway and swap in a gateway key.For example:
2

Enforce your rules

The gateway verifies the key, then checks the request in the order that costs least first: model access (403), then usage limits (429), then your content guardrails (422). Anything refused is blocked here, before it reaches a provider.
3

Route to a provider

It forwards the request to the provider you configured, and can override the model or translate between formats, so an OpenAI-style request runs against an Anthropic model, and vice versa.
4

Return and record

Output guardrails run on the way back, then the response streams to your app. Every request is also published as a trace in your Openlayer project, fire-and-forget, so tracing never slows or breaks a response.Every gateway request becomes a trace in Openlayer

​
What you get

Everything is configured once, in the gateway’s admin portal. There are no redeploys and no code changes in the apps sending traffic. The portal opens on the Dashboard, where you can see what the whole deployment is spending before going into any one policy. The Openlayer Gateway admin portal, open on the Dashboard

Access & keys

Issue and revoke keys for each app or teammate, assign them to groups, and hand out self-serve invite links.

Budgets & usage

Cap spend, requests, or tokens per key, user, or group, and see exactly what’s being consumed and what it costs.

Guardrails

Block or redact PII, stop prompt injection, and screen for unsafe content on inputs and outputs, before they reach a provider or a user.

Model access

Allow or block models per key, user, group, or the whole deployment.

Providers & models

Connect OpenAI, Anthropic, and Azure upstreams, route traffic between them, and call any model from any SDK.

​
FAQ

Almost nothing. You point your client at the gateway’s base URL and use a gateway key instead of a provider key. That’s it. Any client that speaks the OpenAI Responses API or the Anthropic Messages API works unchanged, including streaming.
They serve different jobs and you can use either or both.The gateway exists to enforce your rules on every request, in real time. Tracing comes as a byproduct: because traffic flows through the gateway, every request is also published to your Openlayer project without any SDK in your apps. That makes it the right path when you want governance plus baseline observability across many apps or languages at one chokepoint.The Openlayer SDK runs inside your code. It doesn’t enforce anything at the request boundary, but it captures detail the gateway can’t see from outside: nested spans for retrieval, tools, and sub-calls in a multi-step pipeline, plus any custom metadata. Reach for it when you need fine-grained traces of how a feature actually runs.A common setup is the gateway for governance and baseline traces, plus the SDK in the places that need the deeper view.
The gateway forwards traffic. It doesn’t keep your prompts and responses. It stores hashed API keys, per-day usage tallies, and guardrail event metadata (which guardrail triggered and what kind of entity, not the underlying text).The full request and response content is sent to your Openlayer project as a trace, where your data-retention and access controls apply. Openlayer can run on-premises or as SOC 2 Type II compliant SaaS.
Yes. When you route a request to a provider whose format differs from your client’s, the gateway translates the request, response, and stream between the OpenAI Responses API and the Anthropic Messages API. See Use any model from any SDK.
OpenAI, Anthropic, and Azure OpenAI / Foundry today. Any provider that exposes an OpenAI- or Anthropic-compatible API can be connected. See Connect providers.
The gateway caps how many requests it works on at once and keeps a shallow queue for bursts. Past that, it turns requests away with 503 and a Retry-After header instead of degrading for everyone, so a traffic spike costs you the requests it sheds rather than the deployment.This is a capacity backstop, not a policy: it sits well above normal throughput, and it’s separate from the 429 a usage limit returns. Retrying with backoff is the right response. The admin portal stays available while it’s happening.