LLM

One LLM API for OpenAI, Anthropic and Meta via API

POST /llm/chat/completions has the same body, response and stream as OpenAI's POST /v1/chat/completions, and serves models from three providers. You register each provider key once and the OpenAI SDK works without adaptation: only the base URL and the model change.

The request

POST /api/llm/chat/completions
Authorization: Bearer <merchant token>

{
  "model": "anthropic/claude-sonnet-5-5",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "system",
      "content": "You summarize marketing data in English."
    },
    {
      "role": "user",
      "content": "Summarize the account's week."
    }
  ]
}

What you can do

  • Call OpenAI, Anthropic and Meta (Muse) models with a single body in the OpenAI format.
  • List the available models, with price and limits, in GET /llm/models.
  • Switch provider by changing only the value of model, for example anthropic/claude-sonnet-5-5 or openai/gpt-5.4-mini.
  • Use stream, tools and images on the models that support them.
  • Authenticate with the merchant Bearer only, no x-customer-token.

Rules and limits

  • The gateway only calls a provider with the merchant's own key, registered once in /merchants/credentials/{openai|anthropic|meta}. The provider bills you directly.
  • Without the provider key for a model in the call, the response is 400 llm_credential_missing.
  • max_tokens (or max_completion_tokens) is required on every call.
  • n must be 1 or absent; audio, modalities, prediction, logprobs, top_logprobs and logit_bias are not accepted (llm_unsupported_field).
  • Errors come in the OpenAI error format: {"error": {"message", "type", "code"}}. Branch on error.code.

Main fields

FieldTypeRequiredDescription
modelstringYesModel id from GET /llm/models, in provider/model form.
messagesarrayYesThe conversation, as in OpenAI. role: system, developer, user, assistant or tool.
max_tokensintegerYesRequired (or max_completion_tokens). Up to the account cap and each model's max_output_tokens in the call.
owned_bystringNoIn GET /llm/models: the model's provider (openai, anthropic or meta).
modelsarrayNoFallback: up to 3 models from GET /llm/models, tried in order if model fails before the first token.
streambooleanNoWith true, the response is text/event-stream.

Who it is for

  • AI features in your SaaS

    Summarize reports, write captions or answer customer questions with the model of your choice, without writing one client per provider.

  • AI agents

    Your agent uses the OpenAI SDK pointed at Connect and switches models depending on the task.

  • Comparing models

    Run the same prompt on models from different providers by changing only model, to pick the best cost and quality.

Frequently asked questions

Can I use the OpenAI SDK?

Yes. The body, response and stream are OpenAI's, so the SDK works without adaptation: point the base URL to Connect, use the merchant Bearer and pick the model from GET /llm/models.

Who pays for model usage?

The provider, on your account. The gateway only calls a provider with the key you registered in /merchants/credentials/{openai|anthropic|meta}. Connect records the tokens and cost of each call.

Do I need the x-customer-token?

No. LLM routes use only the merchant Bearer: usage and cost belong to the merchant, not to a customer.

Related

See everything you can do with LLM

See Connect applied to your product.

Try it for 15 days with no card, or walk our team through your use case.