Part I of this series, published on September 11, 2026, was the service-layer story: a Python library that became a platform, then a rewrite in Rust, completed in the second quarter of 2026 by two engineers, that OpenAI says now handles 95% of production requests. Part II, published on October 23, goes one layer down, to the database underneath, Azure Cosmos DB, and to the two problems that layer kept causing: keeping one tenant's traffic from degrading everyone else's, and getting reads back faster. The platform figures Part I published in September 2026 - over 70 million requests per second, more than 500 petabytes, almost 40 regions, over 1 billion weekly users - are the backdrop rather than the news; Part II's own framing puts Habitat at over 1 billion users across 40 geographic regions and more than 500 petabytes.
The container history, reversed twice
OpenAI says it started with a single shared container per account, moved to separate containers for each large service or workload, then came back to shared containers by default once the Cosmos DB team built it an improved serverless mode. That mode lets every physical partition scale up to 10,000 request units per second and introduced partition splits based on request-unit usage, in addition to storage-based splits, which let OpenAI stop monitoring and scaling containers by hand.
OpenAI says it has had no provisioning or noisy-neighbor incidents in the months since adopting the revised shared-container design. Read that claim at its exact scope: it covers those two incident categories. It is not a statement about Habitat's overall availability, and it is not a permanent reliability guarantee. OpenAI gives no adoption date, so 'the months since' is as precise as the claim gets.
Read the 12x as a median
OpenAI reports that Habitat's worldwide median storage-read latency fell from roughly 120 milliseconds to roughly 10 milliseconds during 2026, an approximately 12-fold improvement. The measurement concerns storage reads, not complete ChatGPT responses: it says nothing about model inference or what any individual user waits for.
More than 90% of Habitat requests are reads, per OpenAI. The improvement came from three changes combined: replicating the smallest, most latency-sensitive datasets broadly so data sits near users, an integrated caching layer built into Habitat, and the Rust rewrite of the service layer described in Part I. OpenAI does not isolate how much each change contributed. A median also means half of reads are slower than 10 milliseconds, and OpenAI is explicit about the tail it has not fixed: the slowest requests still require fetching uncached data from the underlying storage, often across distant regions. The post publishes no tail-latency figure to set against the median.
The superlative needs the same treatment. OpenAI says its largest Cosmos DB account stores tens of petabytes and serves more than 20 million queries per second. It believes no other company has deployed Cosmos DB at that scale. The comparative claim is not independently substantiated in the post.
The outages are the useful part
In late 2024, a power outage in a primary Azure region exposed a recovery limitation: OpenAI and Azure discovered that failing over their largest Cosmos DB accounts could take several hours, a scenario neither team had tested at that scale. About a month later, a return to the original region caused another serious outage when clients overwhelmed its database gateway before the underlying partitions had fully moved back: a quirk in the gateway-mode client made every client reconnect to the original region at once, even though only a subset of physical partitions had moved.
Our inference, not OpenAI's wording: a failover procedure that has never been exercised at production scale is not yet a demonstrated recovery capability, and OpenAI is saying plainly that this is what it had. The follow-up since has been joint work with Azure: monitoring at every layer, from Habitat's service layers down to Cosmos DB's physical storage, and testing changes against a production-sized but non-production account. Teams running smaller systems on managed databases should read their own disaster-recovery documents with that sequence in mind.
What to do with this
Nothing here changes what a ChatGPT or API customer can buy, configure or expect contractually. There is no new limit, price or availability claim attached to the post. Its value is diagnostic: when a vendor quotes you a median, ask for the tail; when a platform claims a tested failover, ask when it last ran at full size.
OpenAI's next storage system is an exabyte-scale build for Habitat's bulkiest data, combining Rockset's RocksDB Cloud core with Raft-based replication. It keeps the Cosmos DB document model but uses disaggregated storage and physical replication to reduce the amount of flash capacity needed, in exchange for a lower peak queries per second. OpenAI describes it as in development, not in production. Until it ships and carries production traffic, it belongs in the same category as the 2024 failover plan: stated intent, not demonstrated capacity.
Sources
- [1] OpenAI, 'Rapidly scaling online storage to serve over 1 billion ChatGPT users (Part II)' (published October 23, 2026) - primary source for the Cosmos DB container history and serverless mode, the two late-2024 outages, the read-latency figures, the largest-account claim and the exabyte-scale system; read first-hand October 23, 2026Read source
- [2] OpenAI, 'Rapidly scaling online storage to serve over 1 billion ChatGPT users (Part I)' (published September 11, 2026) - primary source for the September 2026 platform figures and the Python-to-Rust rewrite; read first-hand October 23, 2026Read source