2026-09-03 · Mac · Apple M2
A model scorecard round on M2 · Apple M2 (integrated GPU) · 16 GB: 3 models across 2 task sets. Every round is listed in the model scorecard, with its results side by side in one table.
M2 · Apple M2 (integrated GPU) · 16 GB · llama-cpp · 1 trial per task · 2026-09-03 · gezel a06b1070 · catalog 0.1.52
Three models on an Apple M2 with 16 GB, using native engines 0.1.38. Only general capability and office work were tested. The Quality column was scored afterward by Claude Sonnet 4.6. Qwen 3.8 27B could not be measured: one version needs more memory than this Mac can spare, and the smallest version ran below our minimum speed of 3 tokens per second. This Mac reads long prompts slowly, so tasks were allowed 45 minutes without visible progress before being stopped; that makes this round not directly comparable to others.
General capability
| Model | Size | Tasks passed | Quality | Performance | Context | Memory used |
|---|---|---|---|---|---|---|
| gemma4-12b-q4 (kv: f16) | 12B | 10/11 (some tasks run once — count not rate) | 6.6/10 (3 pieces) | 13.7 tok/s output 100 tok/s prefill | 147K | 10.1 GB |
| ornith1.5-9b-q4 (kv: f16) | 9B | 10/11 (some tasks run once — count not rate) | 6.5/10 (1 pieces) | 9.8 tok/s output 154 tok/s prefill | 132K | 10.1 GB |
| qwen3.5-9b-q4 (kv: f16) | 9B | 9/11 (some tasks run once — count not rate) | 3.8/10 (1 pieces) | 9.2 tok/s output 121 tok/s prefill | 126K | 10.8 GB |
Tasks in this set: 11.
Office and knowledge work
| Model | Size | Tasks passed | Quality | Performance | Context | Memory used |
|---|---|---|---|---|---|---|
| ornith1.5-9b-q4 (kv: f16) | 9B | 12/13 (some tasks run once — count not rate) | 5.8/10 (9 pieces) | 9.8 tok/s output 154 tok/s prefill | 132K | 10.0 GB |
| qwen3.5-9b-q4 (kv: f16) | 9B | 11/13 (some tasks run once — count not rate) | 5.3/10 (9 pieces) | 9.2 tok/s output 121 tok/s prefill | 126K | 10.8 GB |
| gemma4-12b-q4 (kv: f16) | 12B | 10/13 (some tasks run once — count not rate) | 5/10 (9 pieces) | 13.7 tok/s output 100 tok/s prefill | 147K | 10.0 GB |
Tasks in this set: 13.