LLM
Switch LLM providers without changing code via API
The body of POST /llm/chat/completions is OpenAI's for all three providers. To move from OpenAI to Anthropic or Meta, you change the value of model and nothing else: the gateway translates the call for the provider and returns the response in the OpenAI format.
The request
POST /api/llm/chat/completions
Authorization: Bearer <merchant token>
{
"model": "openai/gpt-5.4-mini",
"max_tokens": 1024,
"messages": [
{
"role": "user",
"content": "Summarize the account's week."
}
]
}What you can do
- Alternate between openai/gpt-5.4-mini, anthropic/claude-sonnet-5-5, meta/muse-spark-1.3 and the other models in GET /llm/models.
- Keep the same code for messages, system prompt, tools and stream across providers.
- Store model in your own config and switch without a deploy.
- Check each model's price and limits before switching, in GET /llm/models.
Rules and limits
- model is the id from GET /llm/models, in provider/model form. An id outside the list returns 400 llm_model_not_allowed.
- You must have registered the target provider's key in /merchants/credentials/{openai|anthropic|meta}, otherwise the response is 400 llm_credential_missing.
- On Anthropic, temperature, top_p and top_k are ignored; on Meta, frequency_penalty and presence_penalty are ignored.
- On Anthropic, JSON response_format, audio or file parts and non-function tools return 400 llm_unsupported_field.
- max_tokens cannot exceed the chosen model's max_output_tokens or the account cap.
Main fields
| Field | Type | Required | Description |
|---|---|---|---|
| model | string | Yes | Model id from GET /llm/models, in provider/model form. |
| messages | array | Yes | The conversation, as in OpenAI. role: system, developer, user, assistant or tool. |
| max_tokens | integer | Yes | Required (or max_completion_tokens). Up to the account cap and each model's max_output_tokens in the call. |
| temperature | number | No | Sampling temperature. Ignored on Anthropic. |
| stop | string | No | Stop sequences, a string or a list. |
Who it is for
Avoid provider lock-in
Your SaaS changes provider when price or quality change, without rewriting the integration.
A/B tests of models
Send the same prompt to different models and compare cost and answer, changing only model.
Model per plan
Use a cheap model on the basic plan and a larger one on premium, chosen through the model field.
Frequently asked questions
Do I need to adapt the body when switching provider?
No. The body is OpenAI's for all of them. The gateway translates for Anthropic: system and developer, text, image_url, tools and tool_calls, tool_choice auto and none, stop, max_tokens, stream and usage.
Do some parameters behave differently?
Yes. On Anthropic, temperature, top_p and top_k are ignored. On Meta, frequency_penalty and presence_penalty are ignored. Also check the model's supports_tools and supports_vision in GET /llm/models.
Which error do I get if I forget the new provider's key?
400 with error.code llm_credential_missing, in the OpenAI error format. Register the key in /merchants/credentials/{openai|anthropic|meta}.
See Connect applied to your product.
Try it for 15 days with no card, or walk our team through your use case.
