Model cards · No. 16 of 17
Gemma 4 12B QAT
UD-Q4_K_XLRetiredRetired
62.0Local AI Index / 100
Clean sweep of the standard half; every hard miss was unfinished.
The numbers
Measured, not quoted.
Gemma 4 12B QAT (UD-Q4_K_XL) ranks #16 of 17 on the diego.technology Local AI Index with 62.0/100: 13/20 on twenty graded tasks (one run), 4/10 on the hard half, at 25.5 tokens/s on a 128 GB AMD Strix Halo machine.
- one run so far
- 13/ 20
- one run so far
- on the hard half
- 4/ 10
- on the hard half
- generation · 301 prompt
- 25.5tok/s
- generation · 301 prompt
- to load
- 6s
- to load
Where the 62.0 points came from
- Capability32.5 / 50
- Hard problems8.0 / 20
- Speed12.5 / 15
- Consistency9.0 / 15
Strengths
Where it wins, where it slips.
Twenty graded tasks in six families. Brighter is better. Hatched means a run ran out of output budget before answering — unfinished, not wrong.
- Code67%
- Math75%
- Data100%
- Tools100%
- Context50%
- Reasoning0%
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
Tasks 11–20 are the hard half. Hover a cell for the task.
Every task, as a list
- 01 Python algorithm (interval merge)100%
- 02 Python bug fix (date arithmetic)100%
- 03 Compound-interest math100%
- 04 Tiered tax calculation100%
- 05 JavaScript ratio stress test100%
- 06 SQL: best value per group100%
- 07 Single tool call100%
- 08 Bilingual email → JSON100%
- 09 Long context (~18k tokens)100%
- 10 Predict program outputbudget
- 11 Payment schedule with prepaymentsbudget
- 12 Two-formula penalty comparison100%
- 13 Expression parser, no eval()budget
- 14 JS async retry with backoff100%
- 15 Exact set coverbudget
- 16 SQL window functions100%
- 17 Multi-file bug huntbudget
- 18 Multi-hop long contextbudget
- 19 Multi-step tool use100%
- 20 Logic puzzlebudget
As a coding agent
Six real repo tasks, four harnesses.
Hidden tests decide the score. Minutes are wall-clock for all six tasks.
- Pi
- 5.33/ 6
- 33 min · One task timed out
- OpenCode
- 6/ 6
- 27 min
- dsh
- 6/ 6
- 34 min
- Hermes
- 4.15/ 6
- 27 min · One task timed out
The record
One full run.
A second run is what moves a setup from provisional to verified.
- 2026-09-3013 / 20 · hard 4 / 10