Debugging Agents in Sema4.ai Studio

Sema4.ai Studio provides several tools and logs to help you understand your agent's thought process, identify issues, and resolve them efficiently. This document will guide you through the primary methods for debugging your agents.

When you interact with an agent, it provides a real-time, transparent view into the its decision-making process. The agent's actions are logged step-by-step, allowing you to trace how it arrived at its final response. The agent analyzes your prompt and plans its actions and decides which tools to use. It then executes these actions step-by-step, providing detailed trace for each action taken.

Each action call entry includes:

• Name of the action: The specific tool being called.
• Input parameters: The arguments passed to the action.
• Output results: The outcome or data returned by the action.
• Any errors encountered: A detailed error message if something went wrong.

When the action is triggered, you can immediately see the input parameters and once the action completes, you can view the output results or any errors that occurred.

Action Logs

The detailed, technical logs of an action's execution are accessible from the sidebar. These are the Python execution logs, which contain much more information than what is visible in the chat. Each action you execute is essentially a Python function, and these logs provide comprehensive details from that function.

To access the action logs, click on the Open Logs icon in the side panel, located just above next to Actions section. This will open a new browser window tab displaying the Action Server logs, which lists all of the recent action runs. From this list, you can see the status of each action (Success, Failed).

Each of this entries represents a single execution of an action. Clicking on it will provide information of it's Run input, Run Outpt and Console output. If you want to drill down into the detailed logs, you can click on the "View Logs" button. This view provides a raw Python log, including detailed stack traces, which are invaluable for pinpointing the exact cause of an error.

Advanced Logs in Studio

For more granular debugging, you can access advanced logs within the Sema4.ai Studio. These logs provide a deeper look into the various services that power the agent, allowing you to trace problems to their root cause.

You can access these logs by navigating to the Settings tab in the Studio interface and selecting the Advanced option. Here, you will find several log categories:

• Agent Server Log: This log provides a detailed view of all events related to the Agent Server, which is responsible for the overall agent execution. This log shows the internal workings of the agent's decision-making process, including model calls and the reasoning behind its actions.
• UI (Trace) Log: This log captures events from the Sema4.ai Studio user interface. It's useful for debugging issues related to the Studio application itself, such as rendering problems, unexpected behavior in the chat, or issues with communication between the UI and the backend services.
• Data Server Log: When you are using actions that interact with data, such as those related to Dynamic Data Access, this log provides insights into the data server's operations. It records information about data loading, transformations, and other data-related processes. Errors here can indicate problems with your data source, access permissions, or data processing logic.

Using the Diagnostics Tool

The Diagnostics tool provides a suite of system checks to ensure your local environment is configured correctly for running agents. This helps in troubleshooting issues that aren't related to an individual agent's logic but to the underlying system. You can find this tool by navigating to the Help page. The tool performs several checks:

• System Checks: Evaluates core system resources like CPU and memory to ensure they meet minimum requirements. You'll see a green checkmark for healthy resources and a yellow warning for potential performance concerns.

• Network Checks: Verifies network connectivity to essential Sema4.ai services and CDNs. A green checkmark indicates a successful connection, while a red icon indicates a connection failure.

• Dependency Checks: This section checks the status of your connections to other services, such as Control Room and Snowflake, by indicating if the connection is successful or if the check was skipped because the services are not linked.

You can run these checks to get a holistic view of your system's health. If a check fails, the tool provides details to help you resolve the issue, such as a faulty network configuration or an outdated component.

Setting up Langsmith Tracing

For advanced observability and evaluation, you can integrate external platforms like Langsmith. This allows you to trace and visualize your agent's interactions in a more structured way. This feature allows your to track each LLM interaction as a trace in the Langsmith UI, giving you a detailed view, helping you debug complex scenarios.

To enable Langsmith tracing:

• Navigate to the Settings page.
• Click on the Integrations tab.
• Toggle Langsmith Enabled on.
• Enter the Langsmith project name and your API Key.

Need More Help?

Don't worry if you're still running into issues! We're here to help. For more details on troubleshooting, community support, and how to submit an issue report, please refer to our Get Help guide.