What the benchmark actually ran
The headline number deserves a slow read, because the test behind it is honest and the headline still does more work than the test.
AWS did not put 75,000 agents on a Raspberry Pi. It ran 80,000 agents on a single thread of a server-class AMD EPYC 9R14, with in-memory stores, talking to a simulated model. Each agent ran twenty turns: a streamed completion of about 1.2 KB that calls a tool, the tool result back through the queue, and a second completion to close the turn. Resident memory grew linearly at about 100 KB per idle agent, most of it the conversation histories, since a suspended agent has no task, stack or connection attached.
This measures memory scaling for managed agents in the specified simulated workload, not 80,000 simultaneous model inferences or production throughput on a Raspberry Pi.
Divide 8 GB by that rate and you get the Pi line. The arithmetic is correct. What it measures is narrower: the cost of remembering a conversation, with histories built from 1.2 KB completions. Real agents accumulate tool outputs, file contents, diffs and error logs. As a simple illustration, 8 GB divided by 1 MB per agent gives about 8,000 agents before accounting for operating-system memory, runtime overhead and other state. That is not a measured Infinity capacity. Still a large number, and a different headline. Keep the rate, about 100 KB per idle agent under test conditions, not the headcount it was multiplied into.
Why suspension is a real idea
The problem Infinity addresses is not invented. Agents spend much of their working life waiting: a coding agent starts a build that runs for twenty minutes, a triage agent waits hours for the next pull request, an assistant waits until morning for an approval. In many conventional agent harnesses, a tool call blocks the agent loop, leaving a process and its memory allocated while the tool runs.
Infinity borrows the fix programming languages already use. In Rust, an async function that awaits an operation becomes a state machine that saves only what it needs to resume. AWS applies the same move one level up: at a tool call, the conversation history contains everything needed to continue later. The runtime persists it, discards everything in memory, and lets any process resume the agent when a message arrives. Each run is a bounded slice: load, one completion, persist, dispatch at most one tool call, end.
On serverless platforms that shape has a concrete consequence. A slice never waits on an operation of unbounded duration, so it fits inside a Lambda invocation that exits when the step is done. AWS's example: an agent waiting on a three-day CI pipeline uses no compute until the pipeline reports back. In the AWS reference architecture, the waiting agent needs no active Lambda invocation. Persisted state, queue and timer infrastructure, external tool execution, subsequent Lambda invocations and model calls can still incur costs. AWS provides no dollar-denominated cost comparison.
The questions the authors leave open
To its credit, the post ends with its own unresolved list.
Models are mostly trained on traces where tool calls return synchronously. Whether a model reasons as well when events arrive interleaved is untested, and the post says so. Scheduling is first-come-first-served, so a user-facing thread and a background event handler get the same priority. Compaction, the summarizing step that keeps long histories manageable, is unstudied for large trees of parent and child agents. Event authenticity is open: a runtime that wakes an agent on an incoming message needs a way to establish where that message came from.
There is also a cost the headline skips. A suspended agent uses no compute while it waits, but resuming is not free: the history is stored and reloaded, the queue delivers, a function runs, and the model makes another completion. Infinity reduces the cost of waiting. It does not reduce the cost of thinking, which may be a substantial part of the bill.
What to do with it
Read Infinity as a design worth stealing before it is a product worth adopting. The model fits any stack, including ones that never touch AWS: persist the conversation at every tool call, treat results and events as queue messages, and let the process die in between. Teams whose agents mostly wait, on triage, monitoring or approval work, should try the code, knowing it is a research release looking for feedback.
Everyone else can wait for the missing evidence: a benchmark with a real model and realistic histories, and an answer to how models handle events that arrive while they were asleep. The 75,000 figure extrapolates from roughly 100 KB of resident memory per idle agent in a simulated workload. The model's individual completions were about 1.2 KB; real conversation histories and tool outputs may produce very different memory requirements.
Sources
- [1] AWS Open Source Blog, "Introducing Infinity: a runtime for extreme agent concurrency" (October 23, 2026) - primary source: benchmark method, serverless reference deployment and the authors' open questions, read in fullRead source
- [2] Infinity on GitHub, hydro-project/infinity - the runtime's source repository, linked from the AWS post; Apache 2.0Read source
- [3] AWS News Feed mirror of the same post, published 2026-10-23T17:02:16ZRead source
- [4] Infinity project site, linked from the AWS post (existence and GitHub link only; extra project-site claims not used)Read source
- [5] Dosa AI Tools, Oct 24 brief (secondary roundup that surfaced the post)Read source