S3 Eventual Consistency Edge Cases After Strong Consistency Rollout
Strong consistency doesn't cover concurrent writes, replication lag.
S3's strong consistency rollout on December 1, 2020 fixed read-after-write, the issue everyone complained about. Write an object, read it back immediately, get the right answer, every time, in every region, for free. That part of the story is done. What's left is a handful of edge cases that still trip people up precisely because the 2020 announcement was so sweeping that engineers assumed it covered everything. The 2020 announcement did not cover everything, even though its sweep led engineers to assume it did.
Netflix ran a whole side system called s3mper, backed by DynamoDB, just to track which objects were "really" there versus which ones S3 was still catching up on. Netflix ran a whole side system called s3mper, backed by DynamoDB, just to track which objects were "really" there versus which ones S3 was still catching up on. Cloudera had its own version called S3Guard, same idea. Salesforce was planning around scenarios where eventually-consistent directory listings could feed bad data into Presto-Hive queries, silently producing wrong answers on correct-looking queries. After 2020, all of that scaffolding became dead code. Real engineers deleted real infrastructure because the underlying guarantee finally did what people assumed it always did.
"GET, PUT, and LIST, plus tag, ACL, and metadata changes" is a specific scope. Outside those lines, S3 still behaves like a distributed system, because it is one.
Concurrent writes to the same key and their surprising results
Two writes land on the same key at close to the same time, and one of them starts before the other finishes. S3's own documentation is upfront about what happens next. It picks a winner internally, using last-writer-wins logic, and that winner is not necessarily the write your application "meant" to stick.
The distinction that matters is timing. If Write 1 completes and then Write 2 completes, every read after that point sees Write 2's value, consistently and forever. That's the guarantee working exactly as advertised. The trouble is the in-between moment: a read that fires off while Write 2 is still mid-flight can come back with either value, depending on which side of the race it landed on. Nobody promised you'd like the outcome, only that it would eventually be consistent.
S3 has never supported the kind of mutex-style object locking you'd expect from a database with row-level locks. For years, that meant concurrent-write races were just a fact of life you engineered around with naming conventions or external coordination. That changed, gradually. In August 2024, PutObject and CompleteMultipartUpload picked up If-None-Match, which lets you say "only create this if nothing's there already." A compare-and-swap option, If-Match, followed in November 2024 for the same two operations, giving you optimistic locking based on ETag. CopyObject didn't get either header until October 2025. So depending on which API call you're making, you may or may not have a real tool for this problem right now, and that gap is worth checking against whatever SDK version is sitting in production.
Cross-region replication lag and delete marker replication as the most consequential operational gap
Cross-region replication was never sold as instant. The source bucket acknowledges your write and moves on, and the copy appears in the destination bucket later, on its own schedule, because replication runs asynchronously. The two buckets are not holding hands.
AWS says most replication finishes inside 15 minutes. Fine. But the same documentation says large objects, high object counts, or KMS-encrypted objects can stretch that out to 48 hours. That's a three-order-of-magnitude gap between the typical case and the worst case, and it's exactly the number engineers forget to write into their disaster-recovery runbooks. Building a cross-region failover plan around "15 minutes" and getting hit with the 48-hour tail during an actual incident is not a fun way to learn the difference.
If bounded replication time actually matters for the system being built, S3 Replication Time Control exists for a reason. It's a paid add-on, and what it buys is a number you can actually hold AWS to: 99.99% of new objects replicated within 15 minutes, backed by an SLA guaranteeing 99.9% within 15 minutes across a full billing month. Without RTC, that 48-hour tail is just how the product behaves. Without RTC, that 48-hour tail is just how the product behaves, as a default rather than a bug or an SLA violation. It's the default, and defaults deserve a second look before they become someone's incident postmortem.
CDN and cache layers sitting in front of S3 that the strong consistency guarantee cannot reach
Strong consistency lives at the S3 API surface. Once a request leaves that surface, it's someone else's contract. A CDN caching S3 responses, a malware scanner sitting in the upload path, a thumbnail pipeline writing derived images to a second bucket, none of those inherit S3's guarantee just by being nearby.
The CDN case occurs most often in practice. A PUT to S3 finishes, and the S3 API will hand back the new version on the very next GET. Meanwhile, a CDN edge node somewhere is still holding the old version in cache, perfectly happy to serve it until the TTL expires or someone triggers a manual invalidation. The object store did its job. The cache just didn't get the memo yet.
This is invisible to application code, and that's the dangerous part. If the app reads through the CDN URL instead of hitting the S3 endpoint directly, there's no signal anywhere in that code path telling you the data might be stale. The strong consistency underneath applies to a different layer than the one your request actually touched.
Pipelines that generate derived assets carry the same structural issue. A thumbnail generator or a transcoding job that writes its output into a second bucket has created a brand-new asynchronous hop, and that hop's timing is governed by whatever the pipeline promises, not by S3's read-after-write guarantee. S3 can be perfectly consistent on both ends of that pipeline and the thumbnail can still lag behind the source image by however long the job takes to run.
Operations outside the core GET/PUT/LIST scope where the guarantee is partial or absent
Event notifications are the classic example of a system that predates the whole strong-consistency conversation and just wasn't part of it. S3 promises at-least-once delivery on events, not exactly-once, and it makes no promise at all about order. The retry mechanism that makes "at least once" possible can also hand you the same notification twice for the same object event.
That means downstream consumers, Lambda functions, SQS queues, EventBridge rules, all of them need to be idempotent by design. This has nothing to do with the 2020 rollout; it's a separate contract that was true before and is still true now. Anyone trying to reconstruct an object's current state purely from the order events arrived in is building on sand, because order was never part of the deal.
S3 Tables has its own version of this same pattern. Read-only replica tables can sit in destination table buckets across regions or accounts, and updates land there "typically within minutes." Queryable through any Iceberg-compatible engine, sure, but a query against the replica might not reflect the latest snapshot sitting in the source. "Within minutes" is doing the same job here that "15 minutes typical" does for CRR: it's a friendly estimate, not a number with teeth, unless there's an SLA attached that actually says otherwise.
Then there's scale itself. At S3's volume, an event with one-in-a-billion odds is a Tuesday. AWS has put real effort into formal verification of how GET, LIST, PUT, and DELETE interleave when versioning is on and version stacks run deep, because at this scale, every rare interleaving eventually happens, repeatedly, and needs to be provably handled rather than hoped away.
If S3 Files is in the stack, it adds one more wrinkle. Presenting a general-purpose bucket as an NFS file system stacks three separate contracts on top of each other, with the object API underneath staying strongly consistent, the file system layer running on close-to-open semantics, and the syncing between those two layers being eventually consistent. Anyone building on S3 Files inherits all three, whether they meant to or not.
Reasoning about where the guarantee holds in S3-backed system design
Strong consistency belongs to the S3 object API, for one bucket, in one region. A second region, a CDN, an external pipeline, or a different API surface entirely each sits outside that box, and stepping into any of them means entering a system with its own rules. That's the whole mental model, really. Everything else is detail.
Run through the surfaces one at a time. Single-region GET, PUT, LIST, and metadata operations need nothing extra, they're strongly consistent out of the box. Concurrent writes to the same key call for If-None-Match or If-Match, added across 2024 and into 2025 depending on the API operation involved, ideally enforced at the bucket policy level so the rule can't quietly get skipped. Cross-region replication should be treated as eventually consistent by default, full stop, with S3 RTC as the upgrade if the application genuinely needs a bounded window. Delete markers and version-specific deletes replicating across regions deserve the same skepticism, especially if GDPR deletion requirements or DR automation depend on them landing on time.
CDNs need cache invalidation designed into the write path from day one, not bolted on after the first stale-content bug report. Event-driven consumers need to be idempotent, always, regardless of how reliable the notification system seems on a good day. Anything running through S3 Tables replicas or S3 Files needs to be tested against its actual layered contract.
The pattern across all of these gaps is the same shape every time: a write still in flight, a boundary between S3 and something outside it, or an asynchronous hop somewhere in the chain. None of that undermines the core guarantee. It just means the guarantee has edges, and knowing where those edges are separates a system that degrades gracefully from one that produces a very confusing bug report on a Friday afternoon. Salesforce threw out workaround code after 2020. The 2024 conditional-write additions kept that same trend going, chipping away at the number of edge cases that need manual handling. The direction is clearly toward fewer workarounds over time, but "fewer" isn't "zero," and the gaps that remain are exactly the ones worth knowing by name.
