Model cards · No. 14 of 17
Devstral 2 24B
Q8_0RetiredRetired
70.0Local AI Index / 100
Better on the hard half than the standard half — its misses were genuine errors.
The numbers
Measured, not quoted.
Devstral 2 24B (Q8_0) ranks #14 of 17 on the diego.technology Local AI Index with 70.0/100: 14.76/20 on twenty graded tasks (one run), 7.76/10 on the hard half, at 8.5 tokens/s on a 128 GB AMD Strix Halo machine.
- one run so far
- 14.76/ 20
- one run so far
- on the hard half
- 7.76/ 10
- on the hard half
- generation · 217 prompt
- 8.5tok/s
- generation · 217 prompt
- to load
- 12s
- to load
Where the 70.0 points came from
- Capability36.9 / 50
- Hard problems15.5 / 20
- Speed8.6 / 15
- Consistency9.0 / 15
Strengths
Where it wins, where it slips.
Twenty graded tasks in six families. Brighter is better. Hatched means a run ran out of output budget before answering — unfinished, not wrong.
- Code71%
- Math50%
- Data83%
- Tools100%
- Context100%
- Reasoning67%
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
Tasks 11–20 are the hard half. Hover a cell for the task.
Every task, as a list
- 01 Python algorithm (interval merge)100%
- 02 Python bug fix (date arithmetic)50%
- 03 Compound-interest math0%
- 04 Tiered tax calculation100%
- 05 JavaScript ratio stress test0%
- 06 SQL: best value per group50%
- 07 Single tool call100%
- 08 Bilingual email → JSON100%
- 09 Long context (~18k tokens)100%
- 10 Predict program output100%
- 11 Payment schedule with prepayments0%
- 12 Two-formula penalty comparison100%
- 13 Expression parser, no eval()76%
- 14 JS async retry with backoff100%
- 15 Exact set cover80%
- 16 SQL window functions100%
- 17 Multi-file bug hunt100%
- 18 Multi-hop long context100%
- 19 Multi-step tool use100%
- 20 Logic puzzle20%
The record
One full run.
A second run is what moves a setup from provisional to verified.
- 2026-09-2614.76 / 20 · hard 7.76 / 10