Elastic Storage Allocation for Unpredictable Agent Workloads

Elastic storage allocation for unpredictable agent workloads starts with an isolated disk for each tenant, a sandbox with its own kernel, and billing on active data. For multi-tenant AI agent applications, the storage boundary must match the execution boundary so one customer’s agent cannot see another’s files. Capacity should grow as work unfolds, without asking the agent to predict its needs before it starts.

Why agent storage breaks under fixed provisioning

When a coding agent is asked to fix a bug, it might write one small file. It might instead clone a repository, install a dependency tree, and make artifacts before it finds the fix. The first path needs a few megabytes. The second can need several gigabytes within the same task, and the choice emerges during execution.

Traditional volume sizing assumes someone can estimate an upper bound, reserve that capacity, and pay for it whether it gets used or not. That works poorly when the workload discovers its next step through its previous one. Reserving a large volume avoids some out-of-space failures, but charges for room the agent may never enter. The alternative is guessing smaller and hoping.

How ephemeral sandboxes handle this today — and where that fails

Ephemeral sandboxes avoid retaining a growing workspace by discarding state when a task ends. The next run starts clean, which sounds appealing until “clean” means downloading the same dependencies again. A coding agent that is refactoring a codebase over several interactions needs the changes it has already made and the packages it has installed. Without persistence, the agent has to do the setup work again each time it restarts, which holds up the task. The environment develops an expensive case of amnesia.

Session limits can turn that repetition into an application responsibility. Before execution stops, the agent must checkpoint externally, then restore state and download needed data when it resumes. Those transfers add time and compute to work already made unpredictable by the task itself. Over-provisioning a durable volume can reduce the disruption, but brings back the original capacity bet.

Even “persistent” needs a more precise definition. A filesystem that survives a stop is not the same as a saved filesystem plus memory, and neither is automatically equivalent to a restorable snapshot. Files can still be there after a running process has stopped. What the next invocation gets back determines which parts of the environment still need rebuilding.

How do you prevent tenant data leaks when agent storage persists?

Persistence makes sandbox isolation a prerequisite. A compromised agent in an ephemeral environment threatens that session’s data, while a persistent environment also exposes accumulated state. Plain containers separate processes but share the host kernel. For agent-generated code executed without human review, that shared kernel is not a sufficient boundary between tenants.

gVisor adds a user-space kernel that intercepts most system calls before they reach the host kernel. It strengthens the boundary compared with plain containers, but does not provide hardware virtualization and adds I/O overhead. MicroVMs instead give each workload its own kernel behind hardware virtualization. An exploit must cross additional hypervisor and hardware boundaries to reach the host. This difference affects how safe it is to let an agent use a tenant’s long-lived files inside the sandbox.

Archil places each agent in its own Firecracker microVM and denies outbound access by default until explicitly allowed. This combines separation between workloads with control over where accumulated files can go. Neither control replaces the other. For persistent sandboxes, the recommended posture is deny-outbound with explicit allowances, with credentials injected at the network boundary and kept outside executing code. A private disk is not much comfort if the agent can send its contents anywhere.

What elastic, active-data billing actually requires from the storage layer

Active-data billing requires more than a different line on an invoice. The storage layer must track bytes actually written, expand addressable space as needed, and reduce the billable footprint when data is deleted. That means separating the logical disk the agent sees from the physical capacity allocated underneath. The agent writes to a path. Capacity backs those writes as they arrive, rather than waiting for someone to resize a statically allocated volume.

The billing consequence is concrete in Archil’s approach, which meters active data and expands without pre-provisioning. An agent that writes 200 MB and deletes 180 MB leaves 20 MB to bill under active-data billing, while advance reservation bills for a 10 GB volume. A platform can closely track how much the processor runs while still charging for reserved storage, so each part of the bill needs its own check.

The interface should also fit the agent’s existing tools. Agents are disproportionately trained on bash and file operations, so a filesystem lets them use an interface they already know without learning a custom storage adapter. Atomic rename, mmap, fsync, hard links, and sparse files let ordinary tools operate unmodified. Durability needs a clear stopping point, too. Once fsync returns, writes must be safe enough that the agent can trust the work it has saved.

An S3-compatible API to the same disk lets pipelines and downstream services access working files without a separate copy. For Archil’s bucket-backed disks, writes are durable when fsync returns and synchronize to the bucket in the background. That is a specific durability and synchronization model, with instant cross-client visibility and cross-client file locking unspecified.

Why the ETL shuffle is the storage equivalent of pre-provisioning

Copying a cloud dataset into a sandbox before an agent can inspect it repeats the volume-sizing mistake on the input side. The platform pays for a full copy before knowing which parts the task will use. A dataset can span terabytes while the useful portion is much smaller. Moving the whole thing first makes the cost follow possible access rather than actual work. It is like packing the whole library for an open-book exam.

An S3, GCS, R2, or Azure Blob bucket that is already in use can stay the source of truth while the agent sees ordinary file paths. Data moves when the agent reads it. The bucket stays where it is, and the mount adds an access layer.

This pattern also supports shared project directories for investigation notes and task summaries. Agents in the same pipeline can coordinate through files rather than requiring a separate messaging or state layer for that work. Sharing still needs a defined boundary. A project directory shared by collaborating agents is different from a disk exposed across unrelated tenants. With that boundary intact, reads and writes can follow the work performed instead of the full dataset the agent might have touched.

Forking and checkpointing as the elastic-capacity equivalent for state

Elasticity also applies to alternative versions of a workspace. If each branch needs a copy of the whole disk, attempts that run in parallel cost a lot before they do anything useful. Copy-on-write changes the cost to the differences between branches.

Shanghai Jiao Tong University’s DeltaBox research, first submitted in May 2026 and revised in June 2026, addresses checkpoint and rollback overhead from duplicating whole states. DeltaBox records inter-checkpoint changes to achieve millisecond-level checkpoint and rollback. Its DeltaFS component freezes the writable layer and inserts a new one. Later updates use copy-on-write, while rollback switches layers.

The layer being restored is crucial. DeltaBox provides OS-level checkpointing that can undo executed commands, while an agent framework’s checkpointer captures logical state such as message history and tool outputs. The latter cannot undo filesystem effects. Restoring what an agent remembers is therefore not equivalent to restoring what its commands changed.

Git worktrees offer a related pattern at the version-control layer, with private working directories, indexes, and branches sharing one object store. The arXiv worktree research describes agent sessions running in parallel without a merge conflict reaching main, showing that this can work without a separate copy of the repository for each session. For a durable workspace such as Archil’s, millisecond checkpointing and branching independent of disk size let installed dependencies survive restarts and support parallel attempts. Copy-on-write billing then charges for the branch’s delta.

How current sandbox platforms approach persistent, elastic storage in practice

The useful comparison has three parts, namely isolation, what survives, and what the meter counts. Fork and checkpoint behavior adds a fourth question for exploratory work. These properties are independent. A platform may offer persistent files without active-data billing, or usage-based compute without documenting how storage branches.

Northflank uses microVMs for CPU sandboxes and gVisor for GPU sandboxes, although these details do not name the CPU hypervisor. It offers no fixed session time limit, persistent volumes up to 64 TB, multi-read-write support, and S3-compatible artifact storage. Compute is charged separately for vCPU-hours and GB-hours, with memory priced at $0.00833/GB-hour, leaving active-data storage metering unspecified. Its bring-your-own-cloud option runs sandboxes inside the customer’s VPC, subject to a supported runtime.

Upstash Box names Firecracker and gives each sandbox its own kernel. Boxes auto-pause when idle and wake with their files still there, which suits an environment that stays with one user over time, with gaps between sessions. Billing covers active CPU only, with no memory charge, making this the clearest current example here of metering use. It does not, by itself, establish active-data storage billing or a disk-forking model.

Fly.io Sprites provides persistent, hardware-isolated Linux VMs, with files and installed tools surviving inactivity. Filesystem checkpoints support rollback to a saved point. Fly.io says it bills by the second and charges nothing while idle, which deals with the cost of keeping an environment ready between tasks. The supplied description does not name its hypervisor or establish an active-data storage meter.

For Archil’s persistent sandboxes, Firecracker supplies the per-agent kernel boundary while the disk preserves packages and files across stops and restarts. Storage is billed on active data, grows without pre-provisioning, and has fast checkpoints and copy-on-write branches. Disks can be managed storage or mounts of existing S3, GCS, R2, or S3-compatible buckets without migration. Every disk also has an S3-compatible API.

Vercel Sandbox also names Firecracker as its isolation technology, with session and concurrency limits varying by plan. It fits applications already on Vercel, but the supplied details do not establish restart persistence, active-data storage billing, or checkpoint behavior. Composio Sandboxes makes a different distinction between locations of execution. Its Remote Sandbox preserves Python execution state across calls within a session, while its Local Sandbox runs inside the customer’s VM, container, CI worker, or Kubernetes job. The local option suits sensitive repositories and internal data, but its boundary depends on that customer-controlled environment, and the supplied details do not specify storage billing or fork support.

Matching the billing model to the workload type

The practical choice starts with how the agent works. Short, bursty tasks such as a single script or one-shot transformation fit serverless execution against a disk, with one command running in a fresh microVM. The environment can be disposable while the disk retains useful output. With active-data billing, storage cost follows what the task writes instead of the size chosen for a volume before the task runs.

Long-running, incremental tasks need a different contract. The disk must grow without a restart or manual resize, and installed dependencies must survive interruptions. Platforms without a fixed session cutoff fit this pattern because they avoid forcing external checkpoint-and-restore work. For agents trying several paths in parallel, copy-on-write branches and millisecond-level forks that work at any disk size make each attempt an everyday event instead of a special occasion that calls for a storage budget meeting.

Multi-tenant products add a requirement across every workload pattern. Each tenant needs an isolated disk that other tenants’ agents cannot see, backed by an execution boundary consistent with that separation. Active-data billing per tenant then lets aggregate cost follow actual use rather than multiplying a worst-case allocation across thousands of customers. Shared project directories belong inside the intended collaboration boundary within each customer’s environment.

The old bargain was to size the volume and pay for the reserved maximum. It suited workloads whose needs were known in advance. Autonomous agents find out what they need as they run, so their storage must grow with the work without asking them to know those needs ahead of time.