Webinar: AI Agents in the Wild — Tuesday, January 20 at 9am PT / 12pm ET. Register now!
Build agents
Reasoning Models
Troubleshooting

Troubleshooting Reasoning Models

This page covers common issues and solutions when working with reasoning models in Sema4.AI.

Common Issues

Model Not Appearing in Studio

Symptom: You are trying to add a supported model to Sema4.AI Studio or Control Room but don't see the option in the provider dropdown.

Recommendation:

  1. Verify your Studio, Control Room, or Team Edition App meets the minimum requirements (see Model Compatibility & Benchmarks)
  2. For Team Edition App users, Snowflake provided models are limited to Claude 4 Sonnet or older, non reasoning.

Slow Response Times

Symptom: Agent responses take significantly longer than expected.

Recommendation:

This is most commonly reported with GPT-5 Medium or High reasoning levels, which don't provide significant improvements over Low. We recommend using GPT-5 Low which provides similar accuracy and consistency with 50% lower latency than Medium and 75% lower latency than High.

If you're consuming OpenAI models through their API platform, we support priority processing (opens in a new tab) which guarantees 50 tokens per second or higher throughput for GPT-5/Codex.

To enable priority processing:

  1. Create a new project in the OpenAI API dashboard
  2. In Project Settings, set Default Service Tier to priority
  3. Create an API key for the new project
  4. Create an LLM instance in Sema4.AI Studio/ACE/SPCS using the new API key

AWS Bedrock Permission Errors

Symptom: Error messages when configuring or using Bedrock models.

Solutions: Ensure your IAM role or user has these required permissions:

  • bedrock:ListFoundationModels
  • bedrock:ListInferenceProfiles
  • bedrock:InvokeModelWithResponseStream

Contact your AWS administrator if you're unsure about your API key permissions.

Azure Rate Limits

Symptom: Hitting rate limits or throttling when using OpenAI models (e.g. GPT-5) through Azure AI Foundry.

Recommendation: If you're deploying your agent to Azure, switch to invoice-based billing (opens in a new tab) to unlock 1M–10M token per minute (TPM) limit. See Azure's quotas and limits (opens in a new tab) documentation for details by model and offer type.

Explanation: On pay-as-you-go billing, OpenAI reasoning models like GPT-5 are rate limited to 20K tokens per minute (TPM), or up to 100K TPM after a month+ wait for a rate limit increase. These limits only apply to pay-as-you-go are insufficient for sustaining testing and production workloads, where customers typically need 500k TPM.

Checking Your Azure Agreement Type

While low model rate limits in the Azure Foundry portal the more obvious tell that you're on pay-as-you-go billing, the following commands are the definitive way to check your agreement type.

Install the Azure CLI (opens in a new tab) and run the following commands.

Check your current/default subscription's quota type:

az rest --method get --uri "https://management.azure.com/subscriptions/$(az account show --query id -o tsv)?api-version=2021-04-01" --query "subscriptionPolicies.quotaId" -o tsv

List all your subscriptions with their billing type:

for sub in $(az account list --query "[].id" -o tsv); do
  name=$(az account show --subscription $sub --query name -o tsv)
  quota=$(az rest --method get --uri "https://management.azure.com/subscriptions/${sub}?api-version=2021-04-01" --query "subscriptionPolicies.quotaId" -o tsv)
  echo "$name: $quota"
done

Look for EnterpriseAgreement_2014-09-01 or similar enterprise/MCA-E quota IDs to confirm invoice-based billing. PayAsYouGo indicates pay-as-you-go billing with lower rate limits.

Unable to Deploy GPT-5 Reasoning models on Azure (Verification Required)

Symptom: You're unable to deploy a reasoning model (e.g. GPT-5) on Azure AI Foundry and are prompted to complete verification.

Recommendation:

  • If onboarding and development is time sensitive, consider using models on the OpenAI API platform directly or contact the Sema4 support team for guidance on the best available non-verified model on Azure Foundry as a temporary workaround.
  • Long term, if agent deployment is planned for Azure, complete the verification process in Azure Foundry. In addition, ensure your Azure subscription is on invoice-based billing (opens in a new tab) — models that require verification also require invoice-based billing to obtain a usable rate limit.

Why this matters: Getting access to verified models is critical for successful Sema4.AI agent deployments because they excel at following processes and logic consistently. Without access to verified models, the best available options achieve a pass rate of roughly 40% on the τ²-telecom benchmark compared to 85%+.

Agent Behavior Changes After Enabling Reasoning

Symptom: Agent responses are different from what you expected after migrating from a non-reasoning to a reasoning model.

Explanation: When migrating from non-reasoning to reasoning models, you may notice behavioral differences. In general, reasoning models should improve agent behavior with more structured responses and better handling of complex tasks. However, if you see unexpected behavior:

In our experience, this most commonly happens with the OpenAI o3 and o4 models. For most use cases, we recommend switching to GPT-5 Low, or GPT-5.1 Codex Max Medium for others.

Solutions:

  1. Review the reasoning trace - Examine the chain of thought to understand how the model is processing your request. This helps identify where the reasoning path diverges from expectations.
  2. Review and adjust your agent's runbook instructions if the reasoning trace reveals issues with how instructions are being interpreted
  3. Test thoroughly with your actual workflows before deploying to production
  4. If you need to revert, go to Advanced Options → select v2.0 Architecture → then reselect your original model

Getting Help

If you're still experiencing issues after trying these solutions:

  1. Check the FAQ for additional guidance
  2. Review the Model Compatibility & Benchmarks page to ensure version compatibility
  3. Contact Sema4.AI support with details about your issue, including:
    • Studio and Agent Compute versions
    • Model and provider being used
    • Error messages (if any)
    • Steps to reproduce the issue
Model Compatibility & BenchmarksActions Gallery