A benchmark built from other people's finished research

MedChemBench takes 99 problems out of published medicinal chemistry: 117 papers and patents, 761 distinct molecular structures. For most problems the model gets what researchers had at a chosen point in a program — the target, the goals, the compounds tried so far and selected findings — and is graded against what the research later showed. Later experimental results are withheld from the prompt and used for grading.

That design has a boundary built into it. The benchmark tests reasoning from historical evidence, and every problem is drawn from published research, so OpenAI concedes that prior exposure to those sources cannot be ruled out. It cannot measure discovery, because discovery is the part that has not been published yet.

The split inside the score

The overall ranking is tight at the top. GPT-6.2 Astra scored 84.5% at the xhigh reasoning setting; the gaps to GPT-6 Astra and GPT-6.1 Sol are 1.2 and 2.5 points.

The component scores, reported for GPT-6 Astra, tell a less tidy story. Reconstructing molecular structures from drawings and chemical groups earned 95.4% of the available credit. In one example, GPT-6 Astra rebuilt a compound with an 18-atom ring correctly in all ten responses. In the illustrated answers, GPT-5.6 Sol produced a 15-atom ring and GPT-5.6 Luna split the molecule into disconnected fragments; both models failed that reconstruction across all ten responses.

Molecular reconstruction comparison: GPT-6 Astra preserves the correct 18-atom ring, GPT-5.6 Sol produces a smaller ring, and GPT-5.6 Luna produces disconnected fragments
Image: OpenAI (MedChemBench announcement)

Selecting compounds and predicting numerical properties earned about half the available credit. Those are the two components closest to the working decision in medicinal chemistry: given several candidates and conflicting properties, which one do you make next, and what will it do? OpenAI notes the categories use different tasks and scoring rules, so the percentages are not directly comparable as difficulty measures. Even with that caveat, 95.4% on structure against about half on selection and prediction is a specific, useful finding. It is not a rounding error in an otherwise uniform result.

What the test cannot say

OpenAI lists the limits itself, and they deserve to be read at the same size as the headline. Each response was a single attempt, with no tools, no feedback and no new experiments. The problems cover reasoning from historical evidence, not clinical development and not prospective discovery. And the field is eleven GPT models: with no rival system tested, MedChemBench currently ranks OpenAI's models against each other, not against the field.

None of that makes the benchmark useless. As a diagnostic it separates "can read the molecule" from "can choose the compound" within its own tasks, and that separation is the finding worth keeping. What it does not support is the shorter headline version, in which an 84.5% score becomes evidence that AI is ready to run drug discovery. OpenAI says in its own post that the results do not establish whether models can discover new drugs or improve laboratory decisions. Readers should hold the number to the same standard.

Sources

  1. [1] OpenAI, "Can AI help scientists identify promising drug candidates?" (MedChemBench announcement), November 20, 2026Read source