Cluster and node requirements

What your Kubernetes cluster and its nodes must provide before the platform can run, why each requirement exists, and the limits that come with them.

Audience: IT (the enabler)

This page is deliberately platform independent. It describes what the cluster must look like whatever you build it on. For the AWS specifics, and for Terraform that provisions this shape for you, see Deploy on AWS EKS.

Why this is not any Kubernetes cluster

Earlier versions installed into a cluster you already had. Version 3 does not, and the reason is the sandbox.

Agent code runs inside a hardware-isolated guest, not a normal container. Each run gets its own kernel and its own filesystem view, with no path back to the host beyond the workspace snapshot it was handed. That guarantee is what makes it safe to let an agent execute code against your data, and it is the foundation of the isolation described in Safe execution and isolation.

Delivering it puts three requirements on the node itself:

• Nested virtualization. The sandbox starts a virtual machine per run, so the node's kernel must expose KVM.
• A sandbox runtime class. Kata Containers must be installed on the node and registered with containerd before any workload lands.
• A copy-on-write filesystem. The workspace layer builds each run's filesystem from shared, reflinked snapshots, which needs btrfs on a dedicated block device.

None of these can be added from inside the cluster after the fact. A node that joins without them cannot run agent work, and the failure surfaces late, as sandbox runs that never start. That is why node preparation runs before the node joins the cluster.

Kubernetes requirements

• Kubernetes 1.36 or newer. On AWS that means an EKS 1.36 cluster.
• A working Metrics Server. The platform depends on it, so kubectl top must return data.
• A CSI driver for the platform's block storage.
• An ingress controller or load balancer controller that you operate.
• A secrets operator for delivering credentials into the cluster.
• Two namespaces: one for the application, one for bootstrap work.

Node requirements

• Architecture: x86_64 only. There is no arm64 build.
• Nested virtualization: required, and it must be enabled explicitly. On most clouds this is gated by instance family rather than requiring bare metal, and it is often off by default even on a family that supports it.
• Sizing: 16 vCPU and 64 GiB minimum, which supports a small workspace, and 32 vCPU and 128 GiB recommended for a medium one. The node hosts the whole application plus every concurrent sandbox run, so size it for both. Sandbox concurrency is bounded by a configured runner pool, not by autoscaling, which makes the node the hard ceiling on how much agent work can run at once.
• Headroom: size against allocatable capacity, not the instance label. Cluster add-ons and system DaemonSets are deducted first, and the platform's own baseline requests come out of what is left before any sandbox run starts.
• A node label identifying the node as the application node. Every workload is pinned to it.
• One sandbox worker per node. Pod anti-affinity enforces this, and each run's pods are pinned to the same node as the worker that started them.

Block device requirements

The node needs a dedicated block device, separate from the root volume, formatted btrfs and mounted at the platform's data root.

This volume is a cache, not the system of record. Durable data lives in your object storage bucket. The workspace filesystem is materialized onto the node on demand, and the btrfs volume holds those materializations plus the image and blob caches that make them fast. Losing the volume costs you a cold cache, not your data.

That framing matters for two decisions. Size the volume for working set and cache hit rate rather than for total data volume, and treat it as replaceable rather than something to back up.

It must satisfy all of the following:

• Reflink support. Copy-on-write cloning has to work. This is what makes per-run workspaces cheap instead of a full copy each time, and it is the main reason the filesystem must be btrfs rather than whatever the node ships with.
• Shared mount propagation. The data root is mounted shared, because the kubelet only accepts bind mounts propagated from a shared source. Propagation cannot be changed by remounting, so it is a distinct preparation step.
• Compression and the subvolume options the platform's workspace layer expects.
• A persistent mount that survives reboot.

Size it generously. The sandbox keeps sizeable base-image and bundle caches here alongside materialized workspaces, and a volume that is too small shows up as slower runs rather than as an error.

Node preparation

Preparation is not a DaemonSet. It runs as part of node bootstrap, before the node is allowed to join, and the kubelet is gated on it: the node only registers once preparation has written its completion marker.

The stages are:

1. Validate the host. Architecture, and the presence of the KVM device.
2. Install host packages.
3. Install the sandbox runtime. Download and verify the Kata Containers release, then register it with containerd as a runtime class.
4. Configure the runtime for this platform's kubelet layout.
5. Prepare the filesystem. Format the data device btrfs if needed, mount it with the required options, make the mount shared, and create the data-root directory layout with its ownership.
6. Verify, then write the completion marker.

If bootstrap fails, it reports the failing stage to the console and the system log, so the node's console output is the first place to look.

Network egress

The deployment needs outbound HTTPS to a small set of Sema4.ai services for licensing, image and chart distribution, and updates, plus the hosts that serve the sandbox runtime during node bootstrap. It also needs egress to everything you connect it to: model providers, MCP servers, integrations, and data sources.

Unrestricted outbound HTTPS is the simplest configuration and what we recommend. For the explicit allow-list, see Network endpoints.

Caveats and limits

Know these before you commit to a design:

• Single node, and no horizontal scaling. This release runs the application on one node. You cannot add nodes to serve more load, and there is no multi-node or multi-AZ topology for the application tier. Scale by sizing the node, not by adding nodes.
• No high availability. Because there is one node, losing it means the workspace is unavailable until a replacement is prepared and joins. Your data is safe in the database and object storage, so this is an availability limit rather than a durability one, but plan your recovery time around it.
• No autoscaling. Neither the cluster autoscaler nor Karpenter is part of the deployment. Sandbox concurrency is a fixed pool size, not a function of load.
• x86_64 only.

What this article will cover

• The validated add-on set and versions
• Node sizing against a target concurrency, with the per-run CPU and memory budget
• What the cache holds, and how a cold node behaves while it refills
• Recovery procedure for a lost node
• The full data-root directory layout and its ownership model
• The complete egress allow-list
• How to verify a prepared node before installing
• Troubleshooting a node that failed preparation