Start with the footnote
The release is four paragraphs of product description and one performance sentence: in joint testing, persistent KV cache delivered up to 56x faster TTFT at 130,000-token prompts. Its footnote points to IBM Redbook MD260021, "Context without Limits," published June 5, 2026 and last updated August 20. The desk read the Redbook directly; the detailed measurements below are from MD260021 itself. Nothing in the November launch establishes a new benchmark. What is new is the packaging: a combination of servers, storage software, inference framework and networking presented as one solution.
That distinction is worth keeping, because a reference architecture and a product are different promises. The Redbook shows what one carefully assembled rig did in June. The launch asks buyers to read it as what this combination will do in their data center.
What 56x is measured against
The 56x figure comes from one comparison at 130,000 tokens. In the Redbook's test figures, full prompt recomputation took 32.14 seconds before the first token, while reading the cached state from shared storage took 0.570 seconds. Dividing the two produces approximately 56.4x. That supports the announcement's "up to 56x faster TTFT" claim for this particular comparison. It is not a 56x improvement in overall inference throughput, nor a whole-system speedup.
Two conditions shape the number, and both are stated in the Redbook. The first is the baseline. At 130,000 tokens, the uncached system spent 32.14 seconds recomputing the prompt before producing a token; the cached system spent 0.570 seconds reading the stored state back. It is the largest gap available to measure: any working cache, in any tier, removes most of that recomputation. The comparison shows the value of having a persistent cache at all, priced against the cost of having none.
The second condition is the cache hierarchy. The GPU-memory and system-DRAM cache tiers were disabled to isolate the shared-storage tier. This is not a normal production configuration: it deliberately removes the faster cache layers that would ordinarily serve eligible requests before shared storage is needed. In a production system those tiers stay on, and they are faster than shared flash. A context that would have been served from GPU memory or DRAM never reaches the storage tier, and a context served from storage is being compared, in practice, against those tiers' miss rate — not against 32 seconds of recompute. The 56x figure answers a clean experimental question. It is not the speedup a buyer with a normally configured inference stack should budget for.
A separate 200-request test provides a different performance measure. The Redbook reports throughput increasing from 0.19 to 4.26 requests per second, approximately 22.4x, while total processing time fell from 1,048.56 to 46.94 seconds, approximately 22.3x. The workload used 28 concurrent connections and 100 unique prompts, totaling 24 million tokens and 825 GB of cache, with the cache starting empty. These are throughput and total-time results, not another measurement of the 56x TTFT figure. Reuse is the whole mechanism: a cache earns its cost only when contexts repeat.
The constraint the test actually found
The Redbook's more durable finding is about bandwidth, not the multiplier. The eight-node storage system could read at up to 315 GB/s, while the client-side test network topped out at 80 GB/s. In that configuration, the network path imposed a lower ceiling than the storage system's reported read capability. Adding faster storage alone would not remove that constraint. That is the sizing question a buyer takes from this document: shared cache capacity can be expanded through additional four-, eight- or sixteen-node configurations, but its practical value depends on whether the network can deliver cached context to GPUs at the rate the workload requires. The Redbook also reports a noisy-neighbor test in which other clients generated 200 GB/s of competing network traffic. Under that test condition, throughput fell to 3.6 requests per second, approximately 18% below the clean-network result. It is a useful warning about network contention, not a universal estimate of production performance degradation.
None of this makes the product a bad idea. Long-context and agentic workloads do present the same context again and again, GPU memory is the scarcest resource in an inference cluster, and a shared tier that survives beyond one node's memory is a sensible place to keep state. What the launch has not shown, yet, is a number measured the way a customer would run it: cache tiers on, a production prompt mix, and a named deployment. The release offers no price, no customer and no availability date. Until one of those appears, the honest summary of the 56x is the footnote's: June's test, November's box.
Sources
- [1] Supermicro, "Supermicro Launches KV Cache Solution with IBM and NVIDIA Demonstrating Significant Inference Performance Benefits" (PR Newswire, November 5, 2026; read first-hand at Gate 1: the four components, the single performance sentence — in joint testing with a 120-billion-parameter LLM, persistent KV cache delivered up to 56x faster TTFT at 130,000-token prompts — footnote [1] pointing to the IBM Redbook, and no price, named customer, order or availability date)Read source
- [2] IBM Redbooks, "Context without Limits: A High-Performance KV Cache Platform for Large-Scale AI Inference" (MD260021; read first-hand in full at Gate 1: recompute TTFT of 32.14 seconds at 130,000 tokens against a cached range topping out at 0.57 seconds, with the Redbook itself stating a 56x speedup at the 130k mark; G1 GPU-memory and G2 system-DRAM cache tiers disabled by setting their capacity to zero, isolating the G4 storage tier; the 200-request run at 28 concurrent connections drawn from 100 unique prompts representing 24 million tokens and 825 GB of KV cache, starting from an empty cache, with throughput rising from 0.19 to 4.26 requests per second and total time falling from 1,048.56 to 46.94 seconds; the noisy-neighbor test with four clients generating 200 GB/s of competing traffic, holding 3.6 requests per second, an 18% reduction; client-facing bandwidth up to 80 GB/s against storage read throughput up to 315 GB/s; the test rig of one client node with four NVIDIA RTX PRO 6000 GPUs, an eight-node Petascale cluster, three Spectrum-X SN5600 switches, Dynamo v0.9.0+, vLLM v0.14.1 and openai/gpt-oss-120b; its version history records content updates between June 10 and August 11, 2026)Read source
- [3] Unite.AI, "Supermicro Launches Shared KV Cache Storage for Long-Context AI" (November 5, 2026; the route for the Redbook's June 5, 2026 publication date and August 20, 2026 last-update stamp, which the Redbook's own HTML version history does not display)Read source