Lab Images Handshake Says Scientists Find Straightforward
Handshake AI describes VIALS as a benchmark of life-science image interpretation tasks that most PhD-level biomedical scientists would find straightforward. The 161 tasks used by Artificial Analysis cover eight domains and include artifacts such as gel blots, flow cytometry plots, plasmid maps and phylogenetic trees. The task is to draw the scientific conclusion supported by the image, not merely describe what appears in it. The published sources do not provide a directly comparable human pass³ score, so the benchmark cannot quantify how far the tested models fall below human performance under identical conditions.
VIALS-AA tests that claim under working conditions rather than exam conditions. The model does not get the image as a static prompt. It works inside an agent harness with code execution and image inspection tools, the way a deployed system would approach an unfamiliar artifact, and the reported score is the share of tasks passed in all three attempts. That pass³ rule is strict, and it should be kept in mind before treating 64% as a raw accuracy figure. A model that answers correctly twice and slips once scores the same as one that never comes close. The number measures reliability as much as knowledge, which for laboratory use is arguably the right thing to measure.
Three points at the top, nineteen at the open line
The podium is tight enough that the order means little: 64, 63, 61, with a grader's judgment call inside every point. The informative gap is further down.
At launch, Kimi K3.1 Pro led the open-weights entries at 45%, followed by MiMo-V2.6-Pro at 43%. Kimi's score was 19 percentage points below Gemini 4 Carbon's 64%. That gap matters for organizations evaluating open-weights deployment, but the leaderboard does not establish the hardware requirements, operating costs or suitability of any particular local installation.
Refusals add another complication. AA reports that 14 of the 161 tasks triggered a safety refusal from at least one model; seven of those tasks drew refusals from multiple models. Claude Sonnet 5.5 refused six tasks, the highest count reported. A refusal does not supply the requested scientific interpretation, but AA's launch post does not specify how refusals are recorded in the pass³ scoring pipeline. The refusal counts therefore should not be treated as a directly established contribution to each model's final score. For a buyer, the practical point is simpler: on this class of image, some models sometimes will not play.
Similar Scores, Different Costs and Turn Counts
Artificial Analysis reports that Claude Opus 5.5 averaged $1.64 per task at a 61% pass³ score, while GPT-6.1 Sol (max) averaged approximately $0.056 at 59%. That is roughly a 29-fold difference in reported task cost for models separated by two percentage points on this benchmark. The figures describe the configurations AA tested, not a universal price-performance ranking.
AA identifies agent-turn counts as a major contributor to the cost spread. It reports approximately 27 turns per task for Opus 5.5 and 25 for Sonnet 5.5, compared with roughly eight for Astra and Sol. More turns can increase the number of model calls and the tokens consumed, but these averages do not isolate the contribution of turn count from model pricing or token usage. The conclusion the figures support is that the tested configurations produced substantially different costs for relatively close benchmark scores.
Anyone procuring for volume image work should read cost per task, at a stated turn budget, before reading the ranking. The board publishes both, which is more than most benchmarks offer.
AA uses GPT-6 Luna (medium) to grade model responses against reference answers. The evaluated field includes other OpenAI models and a Luna configuration at a different setting. That creates a potential evaluator-dependence concern, but the published sources do not establish that the grader favors OpenAI models or its own model family. The results reflect one model judge applying AA's evaluation procedure, not independent assessment by biomedical scientists. Without a separate human-grading audit or agreement study, readers cannot determine how sensitive the leaderboard is to the choice of judge.
Sources
- [1] Artificial Analysis, VIALS-AA launch post (LinkedIn company page, posted November 6, 2026; read in full): the 161 tasks and eight domains; the Stirrup harness, three runs per task and GPT-6 Luna (medium) as grader; pass³ scoring; the top three scores (64% / 63% / 61%); Opus 5.5 at $1.64 per task against Sol at about $0.056, with the turn counts (about 8 vs about 27 and 25) AA identifies as a major contributor; Kimi K3.1 Pro 45% and MiMo-V2.6-Pro 43%; the 14 refusal tasks and Sonnet 5.5's 6 refusals.Read source
- [2] Artificial Analysis, "VIALS-AA: different paths to scientific image reasoning" (launch article, November 6, 2026; read in full): the launch results are presented as of November 6, 2026, with current results on the leaderboard; tool-call classification across the 19 models in the original test set.Read source
- [3] Artificial Analysis, VIALS-AA leaderboard page (read in full): defines the score as the share of tasks answered correctly in all three repeats (pass³); notes that repeats with an immediate safety block are excluded from turn and tool-call counts; it does not state how refusals are recorded in the pass³ scoring pipeline.Read source
- [4] Handshake AI, "ATLAS Visual Life Sciences (VIALS) Benchmark": the benchmark's purpose, its 161 tasks drawn from experimental-workflow artifacts rather than publication figures, and the authors' framing that frontier models cannot yet interpret these images accurately while PhD-level scientists find the tasks straightforward.Read source
- [5] Lau et al., "VIALS: A Benchmark for Visual Interpretation of Artifacts in the Life Sciences," arXiv:2608.21357 (submitted August 21, 2026; revised September 19, 2026; abstract read): the dataset's construction and the original finding on model performance.Read source