Skip to main content
A provider is an upstream LLM service the gateway forwards requests to. You connect providers once on the Providers page, and the gateway routes traffic to them from there.

​
Add a provider

On the Providers page under Settings, add a provider and fill in:
  • Name: a label you choose, such as openai. Routing rules refer to providers by this name.
  • Base URL: the provider’s API root, such as https://api.openai.com.
  • Format: the wire protocol and auth the provider expects. One of openai, anthropic, or azure_openai.
  • API key: the provider’s secret. Enter it here, in the portal.
  • API version (Azure only): appended to each request as ?api-version=....
Save your changes. The gateway picks up the new provider right away, with no restart. Connecting a provider on the Providers page
Provider keys are encrypted before they’re stored, and the portal never shows one again after you save it. The page displays a hint instead, the first and last few characters, so you can tell which key is in place. To replace a key, enter the new one over it.
A provider with no key stored returns 503 on every request routed to it, and shows as having no key on the Providers page. Other providers keep serving.

​
Supported formats

azure_openai uses the same wire protocol as openai, with an API version appended to every request.

​
Default routing

With no routing rules, requests pass through by format: an OpenAI-style request (to /v1/responses) goes to your openai provider, and an Anthropic-style request (to /v1/messages) goes to your anthropic provider. To send traffic anywhere else, see Route requests.

​
Azure and Microsoft Foundry

Point the Base URL at the resource path the model lives under, including its prefix (for example .../openai for OpenAI models or .../anthropic for Anthropic models). Use the azure_openai format with an API version for OpenAI models, or the anthropic format for Anthropic models hosted on Foundry.