Filesystem Concurrency Guarantees Every Parallel Agent Deployment Actually Needs

A persistent filesystem workspace that AI agents use across sessions and parallel runs needs to provide three guarantees, per-agent isolation, durable writes, and fork-on-branch semantics. Agents should read shared datasets without sharing mutable working files, preserve completed writes across restarts, and reconcile parallel changes at a deliberate merge gate.

Why classical concurrency tools break when you apply them to long-running parallel agents

Two agents read one shared file, reason for minutes, and write back changes that each considers correct. The changes combine into a wrong but plausible result. Nothing crashes to announce the problem. An agent’s transaction includes a long stretch of reasoning between reading and writing. A mechanism that protects individual file operations cannot, by itself, protect the plan formed between them.

CoAgent, from Lyu et al. at Shanghai Jiao Tong University, identifies three obstacles. Agent transactions span minutes of inference, their read sets are broad and opaque, and writes to live state take effect immediately. Two-phase locking, or 2PL, holds contenders back throughout that reasoning interval. Optimistic concurrency control, or OCC, lets them proceed but discards minutes of inference when a conflict requires an abort and retry.

CoAgent found that both approaches surrender nearly all concurrency gains on contended workloads. Its advisory protocol instead stays within 5% of serial correctness at 1.4× speedup by routing conflicts to the agent’s reasoning. CoAgent recognizes that an agent may repair a plan within the same transaction.

Why eventual consistency is equally dangerous for agent workloads, and where it is safe

Eventual consistency fails on a different clock. A replica or cache can converge after an agent has already built its plan from an older value, and a later update does not automatically revise that plan. Last-writer-wins can then conceal an unresolved collision. “Eventually” is a poor appointment time for correctness.

Distributed training illustrates the danger when two agents must derive their next action from the same shared state but one sees a stale version. The gradient update they make together can be wrong even though each local read was consistent. This is a coordination failure. The same distinction applies to agents whose individually reasonable actions depend on incompatible versions of a workspace.

Eventual consistency should be used for informational or read-mostly layers where agents can reason and act without first seeing the latest write. Recommendation indexes, audit logs, and dashboards can fit that condition. A mutable agent workspace does not. Major cloud object stores are now strongly consistent per key, but eventual consistency still matters in caches, replicas, and some self-hosted stores. Object-store guarantees and filesystem guarantees are not interchangeable labels.

The three guarantees parallel agents actually need, and what each one rules out

Per-agent workspace isolation rules out silent overwrite within the filesystem. Two agents cannot overwrite each other’s working files if neither can access the other’s mutable workspace. Archil provides isolated persistent sandboxes for AI agents, one implementation of this approach that avoids making every workspace edit a 2PL or OCC problem. The boundary comes before the conflict detector. It does not, however, isolate shared external systems by itself.

CoAgent documents why that distinction is essential in a live Kubernetes cluster. Agent A fixes faulty deployments while Agent B reads geo’s current image to get ready for a canary release. Both report success, yet the resulting cluster state matches no serial execution of the two agents. No lock was violated. Strong reads and durable writes alone cannot prevent an incorrect compound result.

Durable writes rule out losing completed work on restart. For this workload, the relevant boundary is fsync. Once fsync returns successfully, the completed writes must survive a restart, rather than depending on an eventual flush from memory.

Fork-on-branch semantics rule out mandatory inference-time aborts for workspace collisions. Each agent works against a separate branch, with changes kept separate during execution and reconciled at a merge gate. That preserves work for deliberate review. It does not promise full serializability, because two file changes can merge cleanly while expressing incompatible decisions.

How per-agent isolation is implemented in practice — worktrees, microVMs, and the Blackboard pattern

Git worktrees separate working files at the software layer. Each agent gets an extra checkout of the same repository, with an independent checked-out branch while sharing the repository’s object store. Edits stay separate until merge. Ownership still needs an explicit contract, because overlapping assignments can create competing changes that the worktree layout cannot adjudicate. Worktrees also provide neither a kernel boundary nor egress control, and do not prevent a process from reading outside its assigned scope.

Agents get sole read/write ownership of named directory subtrees, and the coordination layer enforces those boundaries. Shared interfaces get a designated read-only set and one named writer. Everyone can read the noticeboard. Not everyone gets the marker.

MicroVMs place the boundary below the process that runs the agent. Each agent runs with its own kernel, using hardware virtualization rather than only file permissions or process boundaries. Infrastructure of this kind has borne production load longer than many product pages suggest, which is a more useful maturity signal than branding. For persistent sandbox implementations such as Archil, the relevant architectural fit is keeping an agent’s durable disk with its isolated workspace. A checkable isolation claim names the hypervisor, default egress policy, and location of secrets, because “secure sandbox” is not a specification.

How do you preserve agent files across sessions without preserving broken state?

Mounting a volume gives files a place to live but does not ensure that they are a usable starting point. An agent that returns tomorrow may get last night’s half-finished install or partial write. The files survived. The setup did not. Because agents build on previous state, an incomplete environment can contaminate later work without producing an obvious failure at startup.

Snapshots or checkpoints let a new run start from an accepted point. Persistent paused sandboxes address resumption by retaining the filesystem and a memory snapshot between runs. Git commits address workspace drift, making unrelated edits visible at commit time and providing rollback points. These mechanisms preserve different kinds of state, so they are not substitutes for one another.

The storage layer therefore needs more than a surviving mount. Writes must be durable once fsync returns, and checkpoints must be fast enough to use as part of the usual work. The disk also needs to remain readable by the rest of the stack without a separate copy, so a checkpoint can serve as an inspectable artifact. Recovery should not require moving house first. Checkpoints and branches are only practical if forking them is fast.

Fork-on-branch semantics — why conflicts belong at merge time, not during inference

At launch, each agent forks the shared workspace and works against its own state. A merge gate later reconciles the results. The inference phases cannot overwrite one another’s files because those files belong to separate forks. Unlike OCC, the model does not automatically throw away the entire reasoning interval when two edits collide. The immediate cost shifts to reconciliation, although an incompatible decision may still require more work.

Branching must be cheap enough to become routine. If it takes seconds or minutes, teams have an incentive to skip it and return to shared mutable state. Lightweight checkpoints and branches, as provided by Archil, fit this model by making a pre-operation snapshot and rollback practical without copying gigabytes. A checkpoint that nobody takes is just a recovery plan written in the future tense.

CoAgent identifies the remaining limit as isolation akin to read-committed. Two branches can each be locally valid yet combine into a state that no serial execution would produce. Its Monotonic Trajectory Pre-Order protocol, or MTPO, explores a stronger model with a serialization order fixed at launch and reads filtered by that order. Notifications ask affected agents to revise their plans, while inverse operations undo and reorder misplaced writes.

He and Yu at OpenKedge extend the question through Cognitive Serializability. Retrieved evidence, policy, and delegated authority can change during reasoning, so ordering writes alone does not establish that their derivation was valid. Typed dependency tokens and trusted mediation have been proposed as ways to keep track of those inputs. These are research directions. For most deployments today, fork-on-branch handles file collisions while human or orchestrator review handles semantic conflicts at merge time.

Egress policy and credential handling as concurrency safety properties, not just security properties

Shared credentials give agents a way around filesystem isolation. One agent’s actions can invalidate another’s authorization context mid-inference without touching its disk. The recommended architecture keeps real API keys and reusable tokens out of the agent’s memory and environment. Outbound calls pass through a gateway that enforces policy and injects credentials at the network boundary. Any agent-scoped credential should expire automatically within the longest expected single task run, typically minutes to at most an hour.

Default-deny egress closes another route to shared mutable state. Allowing only the model API, internal forges, and package mirrors restricts which external systems an agent can reach. A private disk is not a private universe. If agents can freely mutate the same external service, their workspace boundaries cannot prevent interference there. Network policy therefore sets part of the concurrency boundary as well as the security perimeter.

Together, credential handling and egress controls limit side effects that the filesystem cannot see, while leaving the serializability of allowed external operations unguaranteed. Stateful Governance for Concurrent Agentic Systems, by Yuxiang Peng and Xiaodi Wu, identifies stale authorization as a separate failure when policy state changes before an effect occurs. Authorization must remain valid both when the agent first plans an action and when that action takes effect.

Giving agents access to large shared datasets without collapsing per-agent isolation

A shared corpus does not require a shared mutable workspace. Agents can read the same dataset while directing every write to their own forks. This separates access from ownership. The merge gate remains the only point where their output can enter shared state, so a large dataset need not be copied per agent merely to preserve isolation.

Object storage creates interface friction because GET, PUT, and DELETE do not provide native append, directory, or file-lock operations. Python and shell-based agents otherwise need custom SDK code to bridge that gap. A POSIX file interface lets them use familiar bash and file tools instead of spending context on bespoke tooling for each source. The filesystem becomes the agent-facing interface to the underlying bucket’s object operations.

A read-only bucket mount can expose S3, GCS, R2, or Azure Blob without migration, ETL, or a separate dataset copy. Archil’s external mounts are one way to take this storage-agnostic approach, while other parts of the stack can still use the bucket’s native object API. Major stores’ strong per-key consistency does not imply cross-client file locking or instant visibility through every filesystem cache. The mount layer’s documented behavior still governs those details. Keeping the mount read-only keeps agents from using it as another workspace where their writes can clash.

How multi-tenant agent products should partition storage to maintain isolation at scale

Tenant isolation requires keeping tenants from interfering with one another’s state. A silo gives each tenant a dedicated runtime, storage, and API endpoint, at the highest cost and strongest separation. It fits tenants whose data cannot share infrastructure. A pool uses shared infrastructure with tenant-ID filtering at the query layer and costs less for low-sensitivity tenants whose rules allow co-residency. These models make different tradeoffs.

Hardcoding a default tenant ID and expecting operators to replace it at deployment undermines either model. At the same time, tenant storage needs to be elastic and metered on active use because a long research task may consume orders of magnitude more workspace than short summarization tasks. Fixed over-provisioning per tenant becomes prohibitive at scale.

A separate, forkable, independently mountable disk per tenant applies the branch model at that boundary. It can support a parallel attempt, a checkpoint before a risky operation, and access from the rest of the stack without a copy. Within it, each agent’s workspace still needs its own boundaries. The three guarantees are fractal, applying at agent, session, and tenant levels.