A benchmark built from messy evidence
Most science benchmarks hand a model a clean chart from a published paper and ask a question about it. BioBasicsBench starts somewhere less tidy. Its 100 tasks come from Ligo scientists' own experiments, and the evidence is the kind a bench scientist actually faces: a gel photograph, a colony plate, a microscopy stack, a plate-reader export, a handwritten note, a short protocol video. Some tasks combine several of those at once, with a ladder chart or a plasmid map supplied as reference material.
The questions are routine on purpose. Read the molecular weight of a DNA sample. Judge an assay. Spot the error. The harder instruction sits underneath several of them: say when the evidence cannot support a conclusion at all. Each answer is graded against a rubric written by the task's author. Each task was reviewed internally by a Ligo scientist other than its author, externally by a PhD-level wet-lab scientist, and through a final internal review. Credit is given in parts, for the conclusion, the measurement, the reasoning, and for noticing that the data are insufficient.
That design is the strongest part of the release. Failed experiments and ambiguous images are where laboratory time actually goes, and a benchmark that only tests clean interpretation measures the easy half of the job.
Read the 66.7% carefully
At xhigh, GPT-6 Astra earned 66.7% of the available credit. Two qualifications change what that number can support.
First, it is mean reward. A model can collect credit for a sound measurement and a partly right explanation while still reaching the wrong final call, and the average will look respectable. OpenAI is explicit about the definition, and readers should hold onto it: 66.7% is not 67 tasks solved out of 100.
Second, the field is closed. All seven models in the table are OpenAI models, from GPT-5.2 up to Astra. There is no Claude, no Gemini, no open-weight entry. Within that closed field, Astra's advantage over GPT-5.6 Sol is substantial: their reported 95% confidence intervals do not overlap. Against GPT-6.1 Sol, the result is less conclusive. Its 62.0% score comes with an interval that overlaps Astra's 60.3–73.1% range, so the table does not establish a clear separation between the two. Calling Astra the best model at reading wet-lab evidence, full stop, goes further than a single-vendor table can carry.
OpenAI's own explanation is measured. It says stronger native multimodal handling may be key, and its examples do show a change in behavior. In one task Astra noticed that a negative-control plate was labeled for kanamycin, not the ampicillin the experiment used, and refused to treat it as a reliable control. GPT-5.6 Sol had taken the empty plate at face value.
The failures are the useful part
The same post then shows Astra getting the consequential call wrong. In a colony-PCR task, Astra grouped the gel bands correctly and then attached the wrong identity to the groups, assuming the larger PCR product marked successful cloning. It proposed lanes 4, 6, 7 and 8. The reference interpretation, built from the controls, selects 5, 9 and 10. OpenAI states the consequence plainly: following Astra's answer would lead to purification of the incorrect plasmid.
In a second example, Astra counted colonies on a plate about 12% above the reference count, merging close neighbors, missing visible colonies and marking empty agar. With plated volume and dilution fixed, that overcount inflates the estimated efficiency of introducing DNA into cells by the same 12%.
OpenAI's own summary line is the right one: higher benchmark scores do not yet mean models can make reliable judgments in the lab. An assistant that observes correctly and still selects wrongly sends the next day of work in the wrong direction, with a confident structured answer attached.
What would settle it
Three things are absent from the announcement as published. The announcement provides aggregate results and selected worked examples, but does not link a downloadable full task dataset, complete grading rubrics, evaluation code or raw per-attempt results that would let an outside group reproduce the benchmark independently. No model from another lab has been scored, so there is no cross-vendor comparison to check Astra against. And 100 tasks, however carefully reviewed, cannot cover every instrument and protocol a working lab uses.
That makes the release a progress report from the company selling the top-scoring model, with unusually honest worked failures inside it. Treat the 66.7% as evidence that OpenAI's newest model reads routine lab evidence better than its own predecessors, at lower token cost. Wait for an outside rerun, and for at least one rival model in the table, before treating it as evidence about the field.
Sources
- [1] OpenAI, “Measuring how AI agents handle everyday wet-lab science”, November 6, 2026 (read in full first-hand; 100 tasks, five domains, the three-stage review process, 37 settings, 18,500 results, the full xhigh table with confidence intervals and output tokens, all three worked failures, and the line that higher scores do not yet mean reliable lab judgments)Read source
- [2] Unite.AI, “OpenAI Introduces BioBasicsBench to Test AI Agents on Wet-Lab Data”, published 2026-11-06T19:55Z (corroborates the November 6, 2026 date, the five domains, the review chain and the failure examples)Read source
- [3] R&D World, “OpenAI's BioBasicsBench tests AI on everyday wet-lab data”, published 2026-11-06T22:43:30Z (corroborates that the evaluation compares only OpenAI models and that Astra used about half GPT-5.6 Sol's output tokens at xhigh)Read source
- [4] Kingy AI, “BioBasicsBench: OpenAI's Wet-Lab AI Results Explained”, November 6, 2026 (bounded source check finding no article-body link in the announcement to a paper, dataset, full rubrics, evaluation code or raw results; also states Kingy did not reproduce the scores)Read source