gezel Gezel Help

Docbench: models for document work

The model scorecard asks whether a model can finish a job on its own. Docbench asks a different question: when DocBlocks hands a model a single piece of document work, how good is what comes back? It runs the same prompts, parsers, and repairs DocBlocks itself uses, so the score describes what you would see in the editor, not a lab version of it.

What it measures

Every model gets the same documents and the same requests, in six groups:

  • Review — the eight review styles, from a general read to resume feedback, fact checking, and plain-language edits. Each test document has known problems written into it, and a reviewer checks which ones the model found, which suggestions were real but small, and which were wrong. Clean documents check that a model does not invent problems.

  • Writing — new content from a brief, plus text added to the end of a document or inserted in the middle of one, and rewrites (shorter, more formal, fixed grammar, bullets to prose).

  • Pictures — choosing what to illustrate, building diagrams of thirteen kinds, drawing simple illustrations, and writing briefs for an image model. Drawings are judged as rendered pictures.

  • Slides — turning a document into a slide deck.

  • Study — summaries, notes, quizzes, and flashcards that stay true to the source.

  • Narration — cleaning up a dictated transcript without changing what was said.

Each answer gets two kinds of marks. Automatic checks catch answers DocBlocks could not use at all: cut off, unparseable, or ignoring the request. An AI reviewer then grades the rest against a written rubric. An answer DocBlocks could not use scores zero.

Reading the score

Docbench is a weighted average of the six groups, from 0 to 100. Review and writing count for a quarter each, pictures a fifth, study aids 15%, slides 10%, and narration 5%.

95% range is how far the score could plausibly move if we had picked different test documents. Two models whose ranges overlap are not reliably different, so they share a band, and the band is the fairest way to read the order.

Usable is the share of answers DocBlocks could show you at all, before any grading.

Suggestions per document breaks general review down by how much each suggestion would change the draft: small fixes, moderate changes, and substantial ones, beside the trivial and wrong suggestions a model added. A model that finds few substantial problems but many trivial ones will feel busy without being helpful.

Scores compare fairly only within one round: the same test set, the same reviewer, and the same version of the prompts. Each round says which it used.

Sometimes a fix to Gezel changes what a model can do, such as a model that could not produce structured answers until Gezel worked around it. Then that model is measured again in a focused re-run under the same tests, reviewer and date, and listed beneath the full round. Its row in the full table says it was re-measured, and its band is its place in the full field.

Results

2026-11-11 · NVIDIA RTX Spark N1X (6144-core GPU, 20-core CPU) · NVIDIA RTX Spark N1X (6144-core Blackwell RTX GPU) · 23 GB · llama-cpp · pack 0.2.1 · judged by claude-opus-5-5

BandModelSizeDocbench95% rangeReviewWritingPicturesSlidesStudyNarrationUsable
1Qwen 3.8 (27B, Q4)27B8179–82728982897486100%
1Qwen 3.8 (27B, Q2)27B7977–81698778907487100%
1Gemma 4 (12B, Q4)12B7977–81708384867288100%
2Qwen 3.6 (35B-A3B, Q4)35B7673–7962897389748699%
2Ornith 1.5 (9B, Q4)9B7371–7550877886737796%
2Gemma 4 (E4B, Q4)8B7371–7552867486698699%
2Qwen 3.5 (9B, Q4)9B7270–7452877284718299%
3Qwen 3.5 (4B, Q4)4B7169–72478577866779100%
3Nemotron 3.5 Bolt (E4B, Q4)8B6967–7250856682688399%
4Granite 4.2 (8B, Q4)8B6663–6838826982677696%
4Gemma 4 (E2B, Q4)2.3B6461–6640746980657297%
4Nanbeige 4.2 (3B, Q4) (re-measured below)3B6159–634982730697894%
5MiniCPM5 (2B, Q4)2B5957–6235746075597597%
5Granite 4.2 (3B, Q4)3B5956–6134746078596798%
5GPT-OSS (20B, MoE, Q4) (re-measured below)20B5854–6222826667703386%
5Qwen 3.5 (2B, Q4)2B5653–5830745175567195%
5LFM3 (1.7B, Q4)1.7B5552–5830687076571392%
6Llama 3.2 (3B, Q4)3B5451–5632665664536894%
7LFM2.5 (2.6B, Q4)2.6B4744–49407250057085%
7Qwen 3.5 (0.8B, Q4)0.8B4340–4520563270466788%
8MiniCPM5 (1B, Q4)1B3735–4014543263344885%

Suggestions per document in general review (moderate-quality drafts):

ModelSmall fixesModerateSubstantialTrivialWrong
Qwen 3.8 (27B, Q4)2.82.00.30.00.0
Qwen 3.8 (27B, Q2)2.81.50.30.80.0
Gemma 4 (12B, Q4)2.51.50.50.30.0
Qwen 3.6 (35B-A3B, Q4)2.71.30.30.30.3
Ornith 1.5 (9B, Q4)2.50.80.32.01.0
Gemma 4 (E4B, Q4)2.31.00.00.80.5
Qwen 3.5 (9B, Q4)1.81.00.80.50.5
Qwen 3.5 (4B, Q4)1.00.30.30.80.8
Nemotron 3.5 Bolt (E4B, Q4)1.80.30.00.51.0
Granite 4.2 (8B, Q4)2.71.30.01.30.0
Gemma 4 (E2B, Q4)1.30.80.01.31.5
Nanbeige 4.2 (3B, Q4)1.80.50.31.31.0
MiniCPM5 (2B, Q4)1.70.30.31.31.3
Granite 4.2 (3B, Q4)0.50.00.00.50.0
GPT-OSS (20B, MoE, Q4)4.00.00.01.01.0
Qwen 3.5 (2B, Q4)1.30.30.01.30.8
LFM3 (1.7B, Q4)3.00.30.04.03.3
Llama 3.2 (3B, Q4)0.81.00.31.51.0
LFM2.5 (2.6B, Q4)2.50.30.01.00.3
Qwen 3.5 (0.8B, Q4)0.70.00.03.30.3
MiniCPM5 (1B, Q4)0.00.00.33.31.0

docbench 0.2.1 · public fixtures 27f9a7ba3ea4 · 32768-token launch context · judge claude-opus-5-5 (prompt v2) · prompts dated 2026-11-10

Focused re-run — 2026-11-12

2026-11-12 · NVIDIA RTX Spark N1X (6144-core GPU, 20-core CPU) · NVIDIA RTX Spark N1X (6144-core Blackwell RTX GPU) · 23 GB · llama-cpp · pack 0.2.2 · judged by claude-opus-5-5

BandModelSizeDocbench95% rangeReviewWritingPicturesSlidesStudyNarrationUsable
3Nanbeige 4.2 (3B, Q4)3B7169–7353837183697997%
3GPT-OSS (20B, MoE, Q4)20B6764–6940837379677597%

Focused re-run of nanbeige4.2-3b-q4, gpt-oss-20b-q4 beside docbench-sweep-0.2.1-2026-11-10, after fixes to how Gezel serves them; bands place them in that round's field · docbench 0.2.2 · public fixtures 27f9a7ba3ea4 · 32768-token launch context · judge claude-opus-5-5 (prompt v2) · prompts dated 2026-11-10

How the reviewer is checked

An AI reviewer is only worth trusting if it is consistent and hard to fool, so before a round is published the reviewer passes three tests: it must give nearly the same marks when asked twice, agree with a second reviewer from a different model family, and mark down deliberately damaged answers (cut short, padded, with invented numbers, or ignoring the request) at least nine times in ten.

Some test documents are kept private and used only in published rounds. If a model does much better on the public documents than the private ones, that is a sign the prompts have been tuned to the test rather than to real work.

Choosing from this

In DocBlocks, the model list ranks models that fit your computer by their docbench score, so the first suggestion is the best measured model your hardware can run. If a model has no score, it has not been measured yet. That is not a verdict either way.

Watch this article as a slideshow · Read it as markdown, for AI agents