On October 23, 2026, OpenAI engineers Jon Lee, Hrachya Hakobyan, and Chaomin Yu published an engineering report on how OpenAI scaled its Habitat storage layer to support ChatGPT 1 billion users [1]. Habitat stores over 500 petabytes of data across 40 geographic regions for ChatGPT and Codex. As user signups surged, OpenAI moved away from isolated database containers and returned to shared multi-tenant clusters in Microsoft Azure Cosmos DB. This shift helped OpenAI end recurring launch emergencies. Paired with a Rust service rewrite and tiered caching, the changes reduced median global read latency by twelvefold, cutting wait times from 120 milliseconds down to 10 milliseconds [2]. Habitat grew from an early Python library into a globally distributed storage engine for ChatGPT [1].
Three Eras of Cosmos DB
OpenAI relied on Microsoft Azure Cosmos DB from the start of ChatGPT, but scaling Habitat required three distinct phases. In the first phase, OpenAI engineers placed all product workloads into a single default container for each Microsoft Azure account. That early setup let OpenAI product teams launch features fast without designing custom storage plumbing. But as ChatGPT grew, logical partition key collisions and noisy neighbors created stability problems. Heavy query surges from one OpenAI tool routinely slowed down unrelated services running on the same Azure hardware [2]. Workloads clashed often.
To shield high-traffic tools from noisy neighbors, OpenAI moved into a second phase by creating separate Azure Cosmos DB containers for specific workloads. Separate containers isolated traffic well, but capacity planning became a heavy chore for OpenAI engineers. Product teams often deployed updates without pre-scaling their Azure Cosmos DB containers in advance, causing launch delays [2].
OpenAI engineers described those launch-day struggles in plain terms: “Product teams would often ship their features to production and forget to pre-scale and we’d sit in SEV rooms on launch day attempting to scale up containers last minute”. To fix the problem, OpenAI engineers and Microsoft Azure developers built traffic rate limits for hot partition keys, prioritized request queues based on service tiers, and upgraded serverless settings so each partition scales up to 10,000 request units per second. With those protections in place, OpenAI entered a third phase by returning to a single shared container. Jon Lee, Hrachya Hakobyan, and Chaomin Yu wrote that “With these changes, we’ve been able to shift back to using one shared container which has significantly simplified the need to do any upfront or continued capacity planning for product teams while preserving isolation between workloads” [2].
Serving 1 Billion ChatGPT Users
Does ChatGPT have 1 billion users across its services today? OpenAI confirmed that Habitat supports products used by a global weekly audience exceeding one billion people located in 40 server territories across multiple continents [4]. Supporting that volume pushed Azure Cosmos DB far past standard Microsoft Azure setups. OpenAI reported that its primary database deployment holds dozens of petabytes while answering more than 20 million queries per second (QPS). As OpenAI engineers wrote, “To our knowledge, we are the only company which has deployed Azure Cosmos DB at this scale” [2]. Running storage for ChatGPT 1 billion users at that rate forced OpenAI to rethink system boundaries.
To prevent a single database glitch from taking down ChatGPT or Codex, Habitat adopted a cell-based architecture [6]. The design divides storage into self-contained cells that cap throughput at 50,000 requests per second. Each cell acts independently. If an invalid query or bad data crashes one cell, only a small fraction of live user sessions feels any impact. The system also permits WebSocket traffic over TCP port 443 so ChatGPT conversations stay connected reliably [1]. OpenAI can now expand overall capacity simply by adding new cells.
When read volume on any single key exceeds 10,000 requests per second, Habitat automatically spins up virtual key replicas over multiple storage nodes to distribute the load evenly [6]. OpenAI built this virtual ring after outages in late 2024, when a local power failure caused multi-hour failover delays and a connection stampede during recovery [2]. Recovery took hours. Large enterprise rollouts, including Uber deploying ChatGPT Work across 30,000 employees, depend on those hardened failover procedures to keep daily business running smoothly without interruption.
Layered Caching and 10ms Latency
Cutting read latency mattered just as much as preventing database crashes. OpenAI and Microsoft Azure deployed a layered read strategy that colocates data closer to users, replicates latency-sensitive datasets, and deploys local caches fed continuously by change data capture. Together with a service rewrite in Rust, these system updates dropped median global read latency from roughly 120 milliseconds down to roughly 10 milliseconds. That twelvefold speedup lets ChatGPT load conversation histories and saved user settings in milliseconds [2].
Habitat organizes memory and storage into three clear performance tiers. The first tier relies on in-process shared memory allocations in Rust, which deliver sub-50-microsecond reads for active token windows and session keys. The second tier uses a custom log-structured merge-tree engine on NVMe flash drives, offering read latencies below 1.5 milliseconds for recent conversation history and cached user turns. Finally, persistent records and conversation archives flow to distributed key-value storage backed by Raft consensus replication [6]. By handling over 98% of live read operations in Rust memory and NVMe flash tiers, Habitat protects Azure Cosmos DB from traffic surges.
Layered caching also separates user traffic from complex analytical queries. Change data capture streams live updates to isolated Rockset instances, so analytical lookups never slow down live chat sessions [3]. But researchers at Judy AI Lab said that the OpenAI disclosure omits granular figures on cache invalidation costs, consistency trade-offs, and cross-region synchronization delays [5].
Scaling 1 Billion ChatGPT Users with Rust
The software running Habitat underwent its own rebuild before reaching its current speed. In 2024, Habitat began as a simple Python client library that gave ChatGPT and Codex backend teams a basic object-and-edge storage interface. But by mid-2025, coordinating updates among dozens of separate client libraries became risky and slow. OpenAI responded by moving Habitat behind a standalone service, centralizing request routing, access control, audit logging, and regional data residency enforcement in a single governed software layer [4].
Even as a centralized service, the Python codebase hit CPU bottlenecks around compression, encryption, and health checks [4]. During the second quarter of 2026, two OpenAI engineers used Codex and GPT-5.5 to rewrite Habitat in Rust. By September 11, 2026, Rust handled 95% of production storage requests, delivering six times greater CPU efficiency and fifteen times greater memory efficiency compared to the older Python Habitat code [3]. OpenAI said that the rewrite stopped event-loop delays and cut server resource costs [4].
To keep latency stable for ChatGPT 1 billion users, Habitat enforces firm rules on database operations [3]. Simple rules worked best. The platform stores every data node alongside its outbound connections to simplify sharding, while banning arbitrary SQL joins and complex graph traversals from the primary query path. Today, Habitat handles over 70 million requests per second in nearly 40 regions, showing how simple API boundaries protect Azure Cosmos DB clusters from unindexed database queries [4].
Future Storage with RocksDB Cloud
OpenAI is already preparing for larger storage volumes beyond its current 500 petabytes. OpenAI engineers Jon Lee, Hrachya Hakobyan, and Chaomin Yu confirmed that OpenAI is building an in-house online storage platform combining the RocksDB Cloud core from Rockset with Raft-based consensus replication to coordinate global data at scale [2].
OpenAI’s experience with Habitat reflects a wider change in artificial intelligence infrastructure. As user bases swell, data storage and retrieval often become bigger bottlenecks than model inference itself [5]. Providing responsive access to over 500 petabytes of conversational state requires narrow interfaces, clear cell boundaries, and close engineering partnerships with cloud providers like Microsoft Azure [1]. By solving these storage bottlenecks directly, OpenAI built a durable base capable of supporting its next billion users [2].
- ONLINE NEWS Subvolts. (2026, October 23). OpenAI Scales ChatGPT Storage to Over 1 Billion Users. [Article Link]
- ONLINE NEWS Newsroom America. (2026, October 23). OpenAI Details How It Scaled Habitat Storage to Serve Over 1 Billion ChatGPT Users. [Article Link]
- ONLINE NEWS Moen, R. (2026, October 23). OpenAI details Habitat storage upgrades for ChatGPT scaling. Blockchain.News. [Article Link]
- WEBSITE DiscoverAI. (2026, September 12). OpenAI Habitat: How ChatGPT Storage Scaled to 1B Users. [Article Link]
- WEBSITE Judy. (2026, October 23). Quickly Scaling Online Storage to Support Over 1 Billion ChatGPT Users (Part 2). Judy AI Lab. [Article Link]
- WEBSITE Nino. (2026, September 12). Scaling Online Storage for 1 Billion ChatGPT Users. n1n.ai. [Article Link]
