Configure LLMs
Agents and Semantic Data Models need a language model to run. In this step you connect one or more model providers to your workspace, then choose a default model for your agents and a dedicated model for SQL generation.
This is the first-time (self-hosted) setup. For the day-to-day reference once you're running, see LLMs in Administration.
LLM configuration is an admin-only task. You'll find it under Configuration > LLMs in the application sidebar.
Supported providers
Sema4.ai connects to your own model provider accounts — you bring the credentials, and the models run under your provider contract. Four providers are supported.
Create the access in your provider first (use the official guides below), then enter the corresponding fields in Sema4.ai:
| Provider | Set up access (provider docs) | Fields to enter in Sema4.ai |
|---|---|---|
| OpenAI | Create an API key (opens in a new tab) | API key |
| AWS Bedrock | Request model access (opens in a new tab) · Create IAM access keys (opens in a new tab) | AWS access key ID, AWS secret access key, region |
| Azure OpenAI | Create a resource & deploy a model (opens in a new tab) | API key, endpoint URL, deployment name (API version optional) |
| Create a Gemini API key (opens in a new tab) · Vertex AI authentication (opens in a new tab) | Gemini API key — or for Vertex AI: Google Cloud project ID, location, service-account JSON |
Make sure the credentials grant access to the specific models you plan to use, in the region where they're enabled. Most setup problems come from a model not being enabled for the account or region.
Allow network egress to the provider
This is the most common cause of LLM setup failures. Your Sema4.ai deployment runs inside your own AWS or Azure network and calls the model provider over the network. If outbound (egress) traffic to the provider's endpoints is blocked, models can't run — the Test button will fail even when the credentials are correct.
Allow outbound HTTPS (TCP 443) from your deployment to the endpoints for the providers you use:
| Provider | Endpoints to allow (HTTPS, port 443) |
|---|---|
| OpenAI | api.openai.com |
| AWS Bedrock | bedrock-runtime.{region}.amazonaws.com, bedrock.{region}.amazonaws.com |
| Azure OpenAI | {your-resource}.openai.azure.com (and *.cognitiveservices.azure.com) |
| Google Gemini | generativelanguage.googleapis.com |
| Google Vertex AI | {region}-aiplatform.googleapis.com, aiplatform.googleapis.com |
Keep the traffic private (recommended for same-cloud)
If your model provider runs in the same cloud as your deployment, route the traffic over the cloud's private network instead of the public internet. This is more secure and usually simpler to authorize:
- AWS → Bedrock: create an interface VPC endpoint (AWS PrivateLink) (opens in a new tab) for service
com.amazonaws.{region}.bedrock-runtimeand enable Private DNS. Calls then stay on the AWS network and no public egress is needed. - Azure → Azure OpenAI: add a Private Endpoint (opens in a new tab) and the
privatelink.openai.azure.comprivate DNS zone, so traffic stays on the Azure backbone. - Google Cloud → Vertex AI: use Private Service Connect (opens in a new tab).
Where to configure egress
- On AWS: allow the destinations in your security group outbound rules and network ACLs, and ensure a route to the internet (NAT gateway) for any public endpoints — or use the VPC interface endpoint above to avoid public egress entirely.
- On Azure: allow the destinations in your network security group (NSG) outbound rules and any Azure Firewall / user-defined routes — or use a Private Endpoint as above.
Use the Test button when adding an LLM as your egress check: if a configuration with valid credentials fails Test, suspect blocked egress or DNS resolution before anything else.
Add a model provider
Step 1: Create access in your provider
Follow the provider's setup guide in the table above to create the API key or credentials, then gather the fields listed in the right-hand column.
Step 2: Add the configuration in Sema4.ai
- Go to Configuration > LLMs and click Add LLM.
- Select the provider, give the configuration a recognizable name, and choose the model.
- Enter the credentials from Step 1.
- Click Test to validate the credentials and model access before saving.
- Save.
Step 3: Repeat for any additional models
Add a separate configuration for each model you want available — for example, one for your agent model and one for SQL generation.
Choose your workspace models
Two workspace-wide settings on the LLMs tab control which models are used by default:
| Setting | What it does |
|---|---|
| Default LLM | Used by any agent that doesn't have a model explicitly assigned. New agents fall back to this. Agents with an explicit model are unaffected. |
| SQL Generation Model | A dedicated model used for natural-language-to-SQL in Semantic Data Models. This overrides the agent's own model for SQL generation only. |
Always set both. If you don't set a Default LLM, agents without an explicit model can't run. If you don't set a SQL Generation Model, SQL generation falls back to each agent's own model, which may not be tuned for SQL.
Recommended models
Pick the stack that matches the provider you've configured. The second value is the reasoning effort to select for that model.
| Use | OpenAI stack | Claude stack |
|---|---|---|
| Agent | gpt-5.1-codex-max — medium | claude-opus-4.6 — medium |
| SQL generation | gpt-5.4 — none | claude-sonnet-5.6 — none |
If you select a model that isn't in the recommended list for SQL generation, the UI shows a warning. For best results, use a model that's been tested for SQL quality.
Maintenance note
If you delete an LLM configuration that was being used for SQL generation, the SQL Generation Model setting is automatically cleared. After removing any LLM configuration, revisit this tab and confirm both the Default LLM and SQL Generation Model are still set.
For adding more models later, see Add LLM configurations.