2026-10-28 · win32 · AMD Ryzen AI 9 HX 370 w/ Radeon 890M
A model scorecard round on AMD Ryzen AI 9 HX 370 w/ Radeon 890M · NVIDIA GeForce RTX 5070 Ti Laptop GPU · 31 GB: 1 model across 4 task sets. Every round is listed in the model scorecard, with its results side by side in one table.
AMD Ryzen AI 9 HX 370 w/ Radeon 890M · NVIDIA GeForce RTX 5070 Ti Laptop GPU · 31 GB · llama-cpp · 3 trials per task · 2026-10-28 · gezel 22fec99111a5ad9796af83a4a1a222236be91006 · catalog 0.3.1
All four suites completed: 129 attempts, exactly three per suite/scenario, 89 raw passes. The eligible subset is 89/116 after 12 required-tool exclusions and one unevaluable authored gate; that last defect was model-authored, not a machine outage. All attempts remain in task performance. There are 59 artifact judgments: 56 included and three excluded for a mismatched A/B rubric, plus 70 tasks without an applicable rubric. Different rubrics are kept separate. Nine deferred canonical browser checks passed on the unchanged saved artifacts. Reviews assessed source and artifacts, not rendered visual polish. Admission-probe speeds remain separate from task medians. See the October 28 Windows Gemma 4 report download for exact axes, rationales, raw scores, exclusions, tokens, GPU telemetry and provenance.
System memory counts this computer's own memory only. What the model keeps on the separate graphics card isn't included.
General capability
| Model | Size | Tasks passed | Performance | Context | System memory |
|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) (kv: f16) | 12B | 29/33 (88%) | 140.9 tok/s output 1,754 tok/s prefill | 149K | 14.2 GB |
Task performance
Task measurements across all attempted trials, including failures. Measurement methods and limits are stated in period labels and round notes. Values are per-trial medians (minimum–maximum; sample count), not admission probes. Wall time includes tools. System RSS and GPU memory are separate peak measurements; do not add their medians. Periods remain separate because hardware performance can change.
| Model | Period | Trials | Output (tok/s) | Prefill (tok/s) | Wall time (min) | Peak system RSS (GiB) | Peak GPU memory (GiB) |
|---|---|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) | All trials · native 0.1.54 | 33 | 126.21 (99.26–148.97; n=33) | 1,927.35 (1,646.20–2,022.87; n=33) | 1.0 (0.5–12.1; n=33) | 14.19 (10.50–20.66; n=33) | 10.36 (10.35–10.40; n=33) |
Artifact reviews
Advisory means out of 10, with eligible artifact counts. Rubrics are separate and are not a combined model score. A dash means no eligible artifact was scored under that rubric. Missing artifacts have no score; excluded reviews do not enter these means. Not applicable means the task had no applicable qualitative rubric. Judge: codex-inline/current-session. Review identity and limitations are recorded with the evaluation.
| Model | Period | HTML quality | No artifact | Excluded reviews | Not applicable |
|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) | All trials · native 0.1.54 | 4.53/10 (9 pieces) | 0 | 0 | 24 |
Tasks in this set: 11.
Office and knowledge work
Task performance
Task measurements across all attempted trials, including failures. Measurement methods and limits are stated in period labels and round notes. Values are per-trial medians (minimum–maximum; sample count), not admission probes. Wall time includes tools. System RSS and GPU memory are separate peak measurements; do not add their medians. Periods remain separate because hardware performance can change.
| Model | Period | Trials | Output (tok/s) | Prefill (tok/s) | Wall time (min) | Peak system RSS (GiB) | Peak GPU memory (GiB) |
|---|---|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) | All trials · native 0.1.54 | 39 | 122.51 (100.95–143.10; n=39) | 1,857.89 (1,733.81–2,087.06; n=39) | 2.6 (0.5–79.6; n=39) | 17.84 (10.50–21.08; n=39) | 10.36 (10.35–10.36; n=39) |
Artifact reviews
Advisory means out of 10, with eligible artifact counts. Rubrics are separate and are not a combined model score. A dash means no eligible artifact was scored under that rubric. Missing artifacts have no score; excluded reviews do not enter these means. Not applicable means the task had no applicable qualitative rubric. Judge: codex-inline/current-session. Review identity and limitations are recorded with the evaluation.
| Model | Period | craftbook-annotated-bibliography | craftbook-powerpoint-deck | craftbook-research-to-document | craftbook-spreadsheet-model | craftbook-week-plan | docblocks-theme-roundtrip | meeting-followup | wikipedia-research-brief | No artifact | Excluded reviews | Not applicable |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) | All trials · native 0.1.54 | 6.42/10 (3 pieces) | 6.58/10 (3 pieces) | 6.11/10 (3 pieces) | 4.33/10 (3 pieces) | 1.25/10 (3 pieces) | 5.33/10 (3 pieces) | 6.42/10 (3 pieces) | 4.25/10 (3 pieces) | 0 | 3 | 12 |
Not published — some tasks could not be measured on this round: Gemma 4 (12B, Q4) (1 task(s) unmeasured)
Tasks in this set: 13.
Engineering work
Task performance
Task measurements across all attempted trials, including failures. Measurement methods and limits are stated in period labels and round notes. Values are per-trial medians (minimum–maximum; sample count), not admission probes. Wall time includes tools. System RSS and GPU memory are separate peak measurements; do not add their medians. Periods remain separate because hardware performance can change.
| Model | Period | Trials | Output (tok/s) | Prefill (tok/s) | Wall time (min) | Peak system RSS (GiB) | Peak GPU memory (GiB) |
|---|---|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) | All trials · native 0.1.54 | 30 | 121.85 (109.44–139.43; n=30) | 1,738.53 (1,236.60–1,947.13; n=30) | 5.2 (0.8–44.2; n=30) | 18.46 (13.77–21.33; n=30) | 10.36 (10.27–10.48; n=30) |
Artifact reviews
Advisory means out of 10, with eligible artifact counts. Rubrics are separate and are not a combined model score. A dash means no eligible artifact was scored under that rubric. Missing artifacts have no score; excluded reviews do not enter these means. Not applicable means the task had no applicable qualitative rubric. Judge: codex-inline/current-session. Review identity and limitations are recorded with the evaluation.
| Model | Period | craftbook-api-contract-review | craftbook-bug-fix-tdd | craftbook-code-review | craftbook-codemod-sweep | craftbook-deep-security-review | craftbook-refactor-module | No artifact | Excluded reviews | Not applicable |
|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) | All trials · native 0.1.54 | 5.00/10 (3 pieces) | 4.22/10 (3 pieces) | 7.17/10 (3 pieces) | 3.44/10 (3 pieces) | 5.11/10 (3 pieces) | 1.89/10 (3 pieces) | 0 | 0 | 12 |
Not published — some tasks could not be measured on this round: Gemma 4 (12B, Q4) (3 task(s) unmeasured)
Tasks in this set: 10.
Complex workflows
| Model | Size | Tasks passed | Performance | Context | System memory |
|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) (kv: f16) | 12B | 10/26 (38%) | 140.9 tok/s output 1,754 tok/s prefill | 154K | 19.3 GB |
Task performance
Task measurements across all attempted trials, including failures. Measurement methods and limits are stated in period labels and round notes. Values are per-trial medians (minimum–maximum; sample count), not admission probes. Wall time includes tools. System RSS and GPU memory are separate peak measurements; do not add their medians. Periods remain separate because hardware performance can change.
| Model | Period | Trials | Output (tok/s) | Prefill (tok/s) | Wall time (min) | Peak system RSS (GiB) | Peak GPU memory (GiB) |
|---|---|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) | All trials · native 0.1.54 | 27 | 108.41 (97.10–142.73; n=27) | 1,731.36 (1,259.77–1,869.46; n=27) | 6.5 (0.8–43.1; n=27) | 19.29 (11.90–21.48; n=27) | 10.27 (10.27–10.27; n=27) |
Artifact reviews
Advisory means out of 10, with eligible artifact counts. Rubrics are separate and are not a combined model score. A dash means no eligible artifact was scored under that rubric. Missing artifacts have no score; excluded reviews do not enter these means. Not applicable means the task had no applicable qualitative rubric. Judge: codex-inline/current-session. Review identity and limitations are recorded with the evaluation.
| Model | Period | craftbook-invoice-run | HTML quality | No artifact | Excluded reviews | Not applicable |
|---|---|---|---|---|---|---|
| Gemma 4 (12B, Q4) | All trials · native 0.1.54 | 5.53/10 (3 pieces) | 2.63/10 (2 pieces) | 0 | 0 | 22 |
Tasks in this set: 9.
October 28 Windows Gemma 4 12B results
Gemma 4 12B completed all four task sets on an RTX 5070 Ti laptop using native 0.1.54: 129 attempts, three per scenario in each suite, with 89 raw passes. Twelve attempts lacked required tools and one had an unevaluable model-authored gate. The reviewed subset contains 89 passes in 116 attempts; the download retains every raw verdict and exclusion reason.
All 59 applicable artifacts were judged, including outputs from failed
attempts. The Artifact reviews tables keep each rubric separate, exclude
three A/B reviews with mismatched axes, and distinguish 70 tasks with no
applicable rubric from missing artifacts. These were unblinded source and
artifact reviews by codex-inline/current-session. Nine deferred browser
checks subsequently passed on unchanged saved artifacts; these short checks
do not establish complete gameplay or visual quality.
The Task performance tables cover all 129 attempts, with per-trial medians, ranges and sample counts for native output and prefill speed, wall time, peak system RSS and peak GPU memory. The overall median native output rate was 121.16 tokens/s. Admission-probe rates remain separate. Exact token counts, GPU utilization, memory samples and fixed-score breakdowns are in the download.
Download the scorecard data, full judging and performance report, and measurement notes. The run identifier retains the October 27 scheduling date; execution occurred on October 28.