A specialist, graded like a junior lawyer
GLM-Legal is Z.ai's first model built for one profession. Where its general models chase broad leaderboards, this one was mid-trained on legal corpora and then put through reinforcement learning across what Z.ai describes as tens of thousands of realistic legal environments, with working lawyers scoring outputs on whether the issues were spotted, the citations sound and the document usable without rework. The test Z.ai chose to headline is not a quiz. Harvey LAB hands the model complete law-firm tasks — the kind with source documents, a brief and a required output format — across 24 practice areas, and passes it only if every step lands.
The result is 19.2%. Read as a school grade, that is a failure. Read against the grading, it is the more useful kind of number: it measures finished work, not plausible work, and finished legal work is exactly where general models have been weakest.
The independent run landed in the same place
Launch benchmarks usually need an asterisk because the vendor ran them. This one got checked the same day. Vals AI, which maintains its own legal benchmark suite, ran GLM-Legal and recorded 19.17% on Harvey's Legal Agent Benchmark, placing it 7th of 79 models. Z.ai reports 19.2%; Vals independently reports 19.17%. The close agreement supports the headline result, although the two figures alone do not establish identical evaluation configurations.
Two other Vals numbers put the ranking in context. On LegalBench, GLM-Legal scored approximately 86.2%. The two scores measure different capabilities under different evaluation designs: LegalBench tests legal reasoning, while Harvey LAB requires complete task deliverables to satisfy every criterion. The difference illustrates why strong results on legal reasoning tests cannot be assumed to translate into reliable end-to-end legal work. On timing, Vals lists a model-level latency figure of 37 minutes 25 seconds; its Harvey LAB leaderboard separately reports approximately 1 hour 12 minutes per evaluation. Neither figure should be presented as a guaranteed production response time.
Where the claims thin out
Z.ai's launch chart reports GLM-Legal at 77.3 on PLawBench, its Chinese legal benchmark, close to Claude Fable 5 at 77.7. That supports the company's narrow parity framing within its own evaluation, but no independent PLawBench replication establishing the same comparison was found — it stays a vendor-reported result. The Harvey number, by contrast, has an independent run beside it; the announcement would have been stronger leading with the verifiable part.
Access is the other limit. There is no public API and no published self-serve tier: firms apply, and Z.ai decides. Z.ai says the model is coming to AutoClaw, the company's desktop agent, which is how most individual practitioners will actually meet it — but that availability was not confirmed at launch. Its weight status is also unresolved: Z.ai's launch directs developers to apply for API access, while Vals labels the model open-weight, and no publicly downloadable GLM-Legal weight release could be confirmed from the launch materials. Buyers should price the dependency accordingly.
The practical angle
Vals evaluated GLM-Legal using a low token-cost assumption: $0.075 per million input tokens and $0.25 per million output tokens. Whether those rates will apply to commercial GLM-Legal API customers remains unconfirmed. Vals' measured $0.15 per Harvey test is the cheapest benchmark cost in its top ten — a benchmark cost, not a deployment quote. The use case the numbers point to is volume drafting: first drafts of contract reviews, evidence summaries and compliance research that a lawyer then checks. That math works only if the human review it still requires is faster than doing the task from a blank page — at a 19% all-pass rate, the product being sold is a draft engine with unusually honest grading, not a substitute associate.
Sources
- [1] Z.ai announcement, October 12, 2026 — via Unite.AI's report on the announcementRead source
- [2] Vals AI, GLM Legal model page, updated October 12, 2026 (independent Harvey 19.17% / 7th of 79, LegalBench, Legal Research Bench, evaluation pricing, latency)Read source
- [3] AI Weekly, "Z.ai Launches GLM-Legal, Scores 19.2% on Harvey LAB Test," October 12, 2026Read source