The drop is the story, not the rank
A single benchmark rank proves little, and SimpleBench is easier to over-read than most. It has just over 200 questions, its test set is mostly private, and the 83.7% human baseline that headlines are measured against was set by nine people. The project does not publish uncertainty ranges for its scores, and the desk is not going to invent one.
The comparison worth making is internal. Same question set, same AVG@5 scoring method, same vendor, adjacent generations. GPT-6.1 Sol scored 82.9%. GPT-6.2 Sol scores 69.9%. That is a 13.0-point fall on one shared test. It does not establish that the two models were served with matched settings or reasoning budgets, and it does not need to: the leaderboard is the record the project's maintainers publish, and on it, the direction is uniform. Astra falls from 83.6% to 77.3%. Astra Pro falls from 86.5% to 79.2%. And GPT-6.2 Sol, at 69.9%, sits below GPT-6 Sol's 73.1% as well as below the 6.1 model it directly replaces. Every GPT-6.2 variant on the board scores below the variant it replaces.
Two benchmarks, two directions
None of this contradicts OpenAI's launch numbers, because the tests measure different things. The GPT-6.2 system card lists Sol at 96.0% on GPQA Diamond, a graduate-level science test, and puts its FrontierCode 2.0 score within half a point of Astra's. SimpleBench was built for the opposite end of the range: short questions about space, time and social situations where the answer is obvious to a person and the trap is in the wording. Its authors argue that memorized knowledge and approximate reasoning are not enough for that kind of question.
Artificial Analysis, the other independent scorekeeper in this story, does not corroborate a general decline. On its Intelligence Index v5.0.1 at maximum effort, GPT-6.2 Astra rose from 57 to 62 against GPT-6 Astra, and GPT-6.2 Sol rose from 55 to 57 against GPT-6.1 Sol. Its separate Coding Agent Index moved the other way for Sol, down three points from 52 to 49, with Astra level at 53. Those indexes run on different scales and different tasks from SimpleBench, so their movements cannot be netted against the 13 points here. What they rule out is the broad claim. GPT-6.2 is not a generally worse generation. On this one public test of everyday reasoning, it is markedly worse than what it replaced.
What to do with a number like this
Do not switch models over one 200-question board. Do treat it as a reason to run your own checks before a routine upgrade, especially if your workload depends on the model following a situation rather than recalling a fact: multi-step instructions with conditions, scheduling and ordering logic, anything where a trick in the wording changes the right answer. Staying on GPT-6.1 Sol while you test changes nothing on the rate card, since the listed token rates are identical; whether the cost per completed task is identical is a separate question the leaderboard cannot answer.
The broader habit is the one this site keeps recommending. A launch chart shows the tests the vendor chose. A public board the vendor did not choose, read carefully and with its limits stated, shows something else. Neither is the whole picture, and the honest reading of this one is narrow: on the only widely watched public test of everyday reasoning, the new Sol is well behind the old one.
Sources
- [1] SimpleBench, project page and leaderboard, including the leaderboard data file the page renders (read directly)Read source
- [2] DEV Community, "GPT-6.2 Astra vs Sol vs Claude Opus 5.5: Sol 6.2 Scores Below 6.1 on SimpleBench," November 22, 2026 (cross-check of leaderboard rows, OpenAI and Anthropic pricing, and the GPT-6.2 system card)Read source
- [3] Artificial Analysis GPT-6.2 launch assessment, November 17, 2026, as reported by AgentsPulse and DEV (Intelligence Index v5.0.1 and Coding Agent Index comparisons)
- [4] Traictory, "GPT-6.2 Astra drops below the human baseline on SimpleBench, and Sol falls 13 points," November 20, 2026 (secondary)Read source