LLM

Switch LLM providers without changing code via API

The body of POST /llm/chat/completions is OpenAI's for all three providers. To move from OpenAI to Anthropic or Meta, you change the value of model and nothing else: the gateway translates the call for the provider and returns the response in the OpenAI format.

The request

POST /api/llm/chat/completions
Authorization: Bearer <merchant token>

{
  "model": "openai/gpt-5.4-mini",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": "Summarize the account's week."
    }
  ]
}

What you can do

  • Alternate between openai/gpt-5.4-mini, anthropic/claude-sonnet-5-5, meta/muse-spark-1.3 and the other models in GET /llm/models.
  • Keep the same code for messages, system prompt, tools and stream across providers.
  • Store model in your own config and switch without a deploy.
  • Check each model's price and limits before switching, in GET /llm/models.

Rules and limits

  • model is the id from GET /llm/models, in provider/model form. An id outside the list returns 400 llm_model_not_allowed.
  • You must have registered the target provider's key in /merchants/credentials/{openai|anthropic|meta}, otherwise the response is 400 llm_credential_missing.
  • On Anthropic, temperature, top_p and top_k are ignored; on Meta, frequency_penalty and presence_penalty are ignored.
  • On Anthropic, JSON response_format, audio or file parts and non-function tools return 400 llm_unsupported_field.
  • max_tokens cannot exceed the chosen model's max_output_tokens or the account cap.

Main fields

FieldTypeRequiredDescription
modelstringYesModel id from GET /llm/models, in provider/model form.
messagesarrayYesThe conversation, as in OpenAI. role: system, developer, user, assistant or tool.
max_tokensintegerYesRequired (or max_completion_tokens). Up to the account cap and each model's max_output_tokens in the call.
temperaturenumberNoSampling temperature. Ignored on Anthropic.
stopstringNoStop sequences, a string or a list.

Who it is for

  • Avoid provider lock-in

    Your SaaS changes provider when price or quality change, without rewriting the integration.

  • A/B tests of models

    Send the same prompt to different models and compare cost and answer, changing only model.

  • Model per plan

    Use a cheap model on the basic plan and a larger one on premium, chosen through the model field.

Frequently asked questions

Do I need to adapt the body when switching provider?

No. The body is OpenAI's for all of them. The gateway translates for Anthropic: system and developer, text, image_url, tools and tool_calls, tool_choice auto and none, stop, max_tokens, stream and usage.

Do some parameters behave differently?

Yes. On Anthropic, temperature, top_p and top_k are ignored. On Meta, frequency_penalty and presence_penalty are ignored. Also check the model's supports_tools and supports_vision in GET /llm/models.

Which error do I get if I forget the new provider's key?

400 with error.code llm_credential_missing, in the OpenAI error format. Register the key in /merchants/credentials/{openai|anthropic|meta}.

Related

See everything you can do with LLM

See Connect applied to your product.

Try it for 15 days with no card, or walk our team through your use case.