LLMs

Manage the LLM configurations available to the workspace — the configured model providers (and their credentials) that agents and SQL generation use. You can list, create, update, and delete configurations, and set the tenant-wide defaults.

All paths are relative to the base URL and require an Authorization: Bearer YOUR_API_KEY header.

Every LLM endpoint is admin only — LLM configurations carry provider credentials and workspace-wide model allow-lists. The configuration object is returned with credentials in plaintext so an admin can replicate a configuration across deployments; treat the responses accordingly.

The LLM object

PublicLlm is returned by most endpoints here.

Field	Type	Description
id	string	LLM configuration ID.
name	string	Display name.
kind	string	Provider kind — one of `azure`, `azure_foundry`, `bedrock`, `cortex`, `google`, `groq`, `litellm`, `openai`, `reducto`.
description	string	Optional description (nullable).
models	object	Allow list mapping provider → list of model names, e.g. `{ "openai": ["gpt-4-1"] }` (nullable).
configuration	object	Provider-specific configuration, including credentials (see below).
created_at / updated_at	string	Timestamps.

The configuration shape depends on kind. For example:

openai — { "openai_api_key": "sk-..." }
bedrock — { "region_name": "us-east-1", "aws_access_key_id": "...", "aws_secret_access_key": "..." }
azure — { "azure_api_key": "...", "azure_endpoint_url": "...", "azure_deployment_name": "...", "azure_api_version": "..." }
litellm — { "litellm_api_key": "...", "litellm_base_url": "..." }

List LLMs

GET /llms

Returns a PaginatedResponse whose data is the list of all LLM configurations. This list is not paged — next is null and has_more is false.

Get LLM

GET /llms/{llm_id}

Returns a single PublicLlm. Returns 404 if no configuration with that ID exists.

Create LLM

POST /llms

Body

Field	Type	Required	Description
name	string	Yes	Display name.
kind	string	Yes	Provider kind (see the list above). An unknown kind returns `400`.
description	string	No	Optional description.
models	object	No	Provider → models allow list. Must not be empty when provided.
configuration	object	No	Provider-specific configuration and credentials. Empty-string values are ignored.

curl -X POST 'https://{your-deployment-host}/tenants/{tenant-id}/api/v2/llms' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Production OpenAI",
    "kind": "openai",
    "models": { "openai": ["gpt-4-1"] },
    "configuration": { "openai_api_key": "sk-..." }
  }'

Returns the created PublicLlm. Invalid or missing credentials for the chosen kind return 422.

Update LLM

PUT /llms/{llm_id}

Partially updates a configuration; absent fields are left untouched. configuration values are merged into the existing configuration, so a request that rotates a single credential leaves the rest intact.

Body

Field	Type	Required	Description
name	string	No	New name.
description	string	No	New description.
models	object	No	New provider → models allow list.
configuration	object	No	Configuration values to merge into the existing set.

The kind is immutable — it can't be changed after creation.

Returns the updated PublicLlm.

Delete LLM

DELETE /llms/{llm_id}

Deletes a configuration. Returns 204 No Content. If the configuration is referenced by a default pointer (see below), that pointer is cleared automatically.

Default LLMs

Two tenant-wide pointers select which configuration is used by default:

default_llm_id — the global default LLM (the fallback model, also used for evaluations).
sqlgen_llm_id — the LLM used for SQL generation.

Get defaults

GET /llms/defaults

{ "default_llm_id": "string", "sqlgen_llm_id": "string" }

A null value means no default is set.

Set defaults

PUT /llms/defaults

Partial update: an omitted field is left untouched, an empty string clears the pointer, and a non-empty value must be the ID of an existing LLM (otherwise 422).

Body

Field	Type	Required	Description
default_llm_id	string	No	ID of the global default LLM. Empty string clears it.
sqlgen_llm_id	string	No	ID of the SQL-generation LLM. Empty string clears it.

curl -X PUT 'https://{your-deployment-host}/tenants/{tenant-id}/api/v2/llms/defaults' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{ "default_llm_id": "ea81f92a-25a3-40c5-bb37-d1a3bb9ad8bc" }'

Returns the updated pointers.

MCP servers Branding