Serverless tool calling: why an agent's tool calls shouldn't hold a VM open

Most agent platforms give each session a sandbox: a VM or container that starts when the agent starts and stays up until the session ends or a timeout fires. Serverless tool calling drops that assumption. Each tool call (a shell command, a file edit, a test run) runs as short-lived compute against a persistent disk, and nothing runs between calls. For most agents that matches how the work actually happens, and it is where agent infrastructure is heading. It is not a fit for every tool, and the exceptions are worth knowing before you switch.

What serverless tool calling means

An agent loop alternates between the model and its tools. The model reads the context and decides on a tool call; the harness runs the tool and returns the result; the model reads that and decides again. The tools that need a machine are the ones that touch files or run programs: bash, read, write and edit, grep, a test runner.

In the session-sandbox model, those tools run inside one long-lived machine. In the serverless model, the harness runs each tool call as its own unit of compute:

1. Start an isolated environment, typically a microVM, with the agent's disk mounted.
2. Run the one command the model asked for.
3. Return stdout, stderr and the exit code, and shut the environment down.

What carries over from one call to the next is the disk: the repository, installed dependencies, build output and the agent's notes. Processes and memory do not.

Why holding a sandbox open is the wrong default

A session sandbox is sized for the moments when a tool runs, but it exists for the whole session, including the time the model spends generating its next step and the time a person spends reading the result.

The sponsor of this publication measured this on its own engineering team's coding-agent sessions: 384 agent loops over 30 days and about 37,000 tool calls, using only timestamps. The median tool call finished in 0.5 seconds and the median model turn took 2.7 seconds. If each loop had kept a sandbox running from its first tool call until 15 minutes passed without one, the sandbox would have been idle 65% of the time it ran, and 88% in the median loop.

That idle time shows up as one of two costs. Either you pay for a machine that is doing nothing, or you set an aggressive timeout and the agent pays for a cold start, or loses its environment, whenever the session pauses.

Why this became practical

Two things changed:

• Machines start fast enough. Firecracker, the microVM monitor behind several agent sandbox products, documents a startup time under 125 ms. When starting an isolated machine costs tens of milliseconds, starting one per tool call is cheap next to a model turn that takes seconds.
• The disk can outlive the machine. If the agent's working directory is a network file system that mounts in milliseconds and commits writes before the call returns, a fresh machine on every call sees exactly the files the last call left.

With both in place, the per-call design gives you isolation per command, parallelism without a pool of sandboxes, and a workspace that doesn't belong to any one machine.

Where it wins

• Agents that mostly wait. Coding, data and research agents spend most of a session on the model or on a person. Per-call compute only exists while a command runs.
• Parallel fan-out. Running five test files, three candidate fixes or a dozen subagent tasks at once is five, three or twelve calls against the same disk, each in its own environment.
• Work that moves. When the workspace is a disk rather than a sandbox, a task can continue from another process, region or machine without copying anything.
• Clean isolation. Each command starts from a known environment. A runaway process or a broken shell state can't leak into the next call.

Where it loses

This is the part that decides whether it fits your agent:

• Long-running processes. A dev server, a file watcher, a database or anything listening on a port has to stay up between tool calls. Per-call compute ends when the command does.
• State in memory. A Python kernel with a loaded dataframe, or a REPL session, has to be rebuilt on every call.
• Long commands. Per-call platforms cap command length; on one such platform, Archil, each call can run for up to 5 minutes. Long builds and test suites need a machine that stays up.
• Reading many files from a cold cache. Each new environment starts with an empty cache, so a tool that walks thousands of files pays for that on every call.
• Very high call rates. If an agent calls tools nearly every second for hours, a machine that stays up can cost less than paying per call.

The practical answer for most teams is both: serverless tool calls for the everyday tools, and a long-lived sandbox started only for the step that needs a server or a long job, with the same disk mounted in each.

What it looks like in practice

Platforms take different routes to the same idea:

• Per-call execution against a disk. Archil's serverless execution runs each exec call in its own Firecracker microVM with a persistent disk mounted, returns the output and shuts the microVM down. It bills per call with execution included, and each call can run for up to 5 minutes.
• Sandboxes that sleep when idle. Fly.io Sprites stop billing for compute within seconds of going idle, and Cloudflare Containers stop billing once an instance goes to sleep. These keep the sandbox model and shrink what idle time costs.
• Session sandboxes with timeouts. Most other sandbox platforms bill a running sandbox per second and end or pause it after a timeout. That gives you the most flexibility per session and leaves the idle time to you.

Whichever route you take, the design question is the same: keep the agent's state on storage that outlives the machine, and treat the machine as something you start for a command and throw away.