2026-08-22 · Mac · Apple M4 Max
A model scorecard round on M4 Max · 64 GB: 11 models across 2 task sets. Every round is listed in the model scorecard, with its results side by side in one table.
M4 Max · 64 GB · llama-cpp + mlx · 3 trials per task · 2026-08-22 · gezel b3be0a7f · catalog 0.1.39
General capability
| Model | Size | Tasks passed | Quality | Performance | Context | Memory used |
|---|---|---|---|---|---|---|
| gemma4-12b-q4 (kv: f16) | 12B | 31/33 (94%) | 6.9/10 (6 pieces) | 48.2 tok/s output 508 tok/s prefill | 99K | 42.2 GB |
| gemma4-12b-q8 (kv: f16) | 12B | 30/33 (91%) | 8.1/10 (3 pieces) | 30.3 tok/s output 506 tok/s prefill | 83K | 42.0 GB |
| btl4-compact-iq2 (kv: q8_0) | 35B | 29/33 (88%) | 7.1/10 (9 pieces) | 80.4 tok/s output 1,120 tok/s prefill | 256K | 14.6 GB |
| ornith1.5-9b-q8 (kv: q8_0) | 9B | 28/33 (85%) | 4.4/10 (5 pieces) | 47 tok/s output 823 tok/s prefill | 256K | 15.9 GB |
| qwen3.5-9b-q4 (kv: q8_0) | 9B | 28/33 (85%) | 5.3/10 (9 pieces) | 62.9 tok/s output 787 tok/s prefill | 256K | 13.1 GB |
| gemma4-e4b-q4 (kv: f16) | 8B | 26/33 (79%) | 5.4/10 (9 pieces) | 96.5 tok/s output 1,442 tok/s prefill | 128K | 13.7 GB |
| qwen3.5-4b-q4 (kv: q8_0) | 4B | 22/33 (67%) | 6.6/10 (9 pieces) | 89.3 tok/s output 1,336 tok/s prefill | 256K | 11.5 GB |
| gemma4-e2b-q4 (kv: f16) | 2.3B | 17/33 (52%) | 4.3/10 (9 pieces) | 144.1 tok/s output 2,574 tok/s prefill | 128K | 6.9 GB |
| lfm2.5-2.6b-q4 (kv: q8_0) | 2.6B | 13/33 (39%) | 3.6/10 (8 pieces) | 168.4 tok/s output 2,125 tok/s prefill | 125K | 4.9 GB |
| qwen3.5-2b-q4 (kv: q8_0) | 2B | 9/33 (27%) | 3.2/10 (9 pieces) | 155.3 tok/s output 3,084 tok/s prefill | 256K | 5.2 GB |
| mistral-7b-q4 (kv: q8_0) | 7B | 4/33 (12%) | — | 76 tok/s output 762 tok/s prefill | 32K | 9.6 GB |
Tasks in this set: 11.
Office and knowledge work
| Model | Size | Tasks passed | Quality | Performance | Context | Memory used |
|---|---|---|---|---|---|---|
| gemma4-12b-q8 (kv: f16) | 12B | 35/39 (90%) | 5.8/10 (24 pieces) | 30.3 tok/s output 506 tok/s prefill | 83K | 41.8 GB |
| gemma4-12b-q4 (kv: f16) | 12B | 34/39 (87%) | 5.8/10 (24 pieces) | 48.2 tok/s output 508 tok/s prefill | 99K | 41.5 GB |
| ornith1.5-9b-q8 (kv: q8_0) | 9B | 34/39 (87%) | 6.5/10 (27 pieces) | 47 tok/s output 823 tok/s prefill | 256K | 14.9 GB |
| gemma4-e4b-q4 (kv: f16) | 8B | 31/39 (79%) | 5.5/10 (22 pieces) | 96.5 tok/s output 1,442 tok/s prefill | 128K | 12.5 GB |
| qwen3.5-9b-q4 (kv: q8_0) | 9B | 31/39 (79%) | 6.2/10 (25 pieces) | 62.9 tok/s output 787 tok/s prefill | 256K | 11.3 GB |
| qwen3.5-4b-q4 (kv: q8_0) | 4B | 30/39 (77%) | 6.2/10 (24 pieces) | 89.3 tok/s output 1,336 tok/s prefill | 256K | 9.9 GB |
| btl4-compact-iq2 (kv: q8_0) | 35B | 27/39 (69%) | 6.2/10 (25 pieces) | 80.4 tok/s output 1,120 tok/s prefill | 256K | 14.1 GB |
| gemma4-e2b-q4 (kv: f16) | 2.3B | 16/39 (41%) | 4.5/10 (20 pieces) | 144.1 tok/s output 2,574 tok/s prefill | 128K | 6.2 GB |
| lfm2.5-2.6b-q4 (kv: q8_0) | 2.6B | 13/39 (33%) | 4.4/10 (27 pieces) | 168.4 tok/s output 2,125 tok/s prefill | 125K | 4.8 GB |
| qwen3.5-2b-q4 (kv: q8_0) | 2B | 10/39 (26%) | 4.1/10 (26 pieces) | 155.3 tok/s output 3,084 tok/s prefill | 256K | 4.6 GB |
| mistral-7b-q4 (kv: q8_0) | 7B | 0/39 (0%) | 1.6/10 (30 pieces) | 76 tok/s output 762 tok/s prefill | 32K | 9.6 GB |
Tasks in this set: 13.