LLM
One LLM API for OpenAI, Anthropic and Meta via API
POST /llm/chat/completions has the same body, response and stream as OpenAI's POST /v1/chat/completions, and serves models from three providers. You register each provider key once and the OpenAI SDK works without adaptation: only the base URL and the model change.
The request
POST /api/llm/chat/completions
Authorization: Bearer <merchant token>
{
"model": "anthropic/claude-sonnet-5-5",
"max_tokens": 1024,
"messages": [
{
"role": "system",
"content": "You summarize marketing data in English."
},
{
"role": "user",
"content": "Summarize the account's week."
}
]
}What you can do
- Call OpenAI, Anthropic and Meta (Muse) models with a single body in the OpenAI format.
- List the available models, with price and limits, in GET /llm/models.
- Switch provider by changing only the value of model, for example anthropic/claude-sonnet-5-5 or openai/gpt-5.4-mini.
- Use stream, tools and images on the models that support them.
- Authenticate with the merchant Bearer only, no x-customer-token.
Rules and limits
- The gateway only calls a provider with the merchant's own key, registered once in /merchants/credentials/{openai|anthropic|meta}. The provider bills you directly.
- Without the provider key for a model in the call, the response is 400 llm_credential_missing.
- max_tokens (or max_completion_tokens) is required on every call.
- n must be 1 or absent; audio, modalities, prediction, logprobs, top_logprobs and logit_bias are not accepted (llm_unsupported_field).
- Errors come in the OpenAI error format: {"error": {"message", "type", "code"}}. Branch on error.code.
Main fields
| Field | Type | Required | Description |
|---|---|---|---|
| model | string | Yes | Model id from GET /llm/models, in provider/model form. |
| messages | array | Yes | The conversation, as in OpenAI. role: system, developer, user, assistant or tool. |
| max_tokens | integer | Yes | Required (or max_completion_tokens). Up to the account cap and each model's max_output_tokens in the call. |
| owned_by | string | No | In GET /llm/models: the model's provider (openai, anthropic or meta). |
| models | array | No | Fallback: up to 3 models from GET /llm/models, tried in order if model fails before the first token. |
| stream | boolean | No | With true, the response is text/event-stream. |
Who it is for
AI features in your SaaS
Summarize reports, write captions or answer customer questions with the model of your choice, without writing one client per provider.
AI agents
Your agent uses the OpenAI SDK pointed at Connect and switches models depending on the task.
Comparing models
Run the same prompt on models from different providers by changing only model, to pick the best cost and quality.
Frequently asked questions
Can I use the OpenAI SDK?
Yes. The body, response and stream are OpenAI's, so the SDK works without adaptation: point the base URL to Connect, use the merchant Bearer and pick the model from GET /llm/models.
Who pays for model usage?
The provider, on your account. The gateway only calls a provider with the key you registered in /merchants/credentials/{openai|anthropic|meta}. Connect records the tokens and cost of each call.
Do I need the x-customer-token?
No. LLM routes use only the merchant Bearer: usage and cost belong to the merchant, not to a customer.
See Connect applied to your product.
Try it for 15 days with no card, or walk our team through your use case.
