← BlogBenchmark · AZ-Focus · 14 October 2026

More correct answers within your budget.

On a local machine, every token a reasoning model writes is time you wait. So we scored our models the way they are used: an answer only counts if it is correct and finished within a time or token budget.

Less wait, more answers: AZ-Focus vs stock on local hardware. +10 points correct within 15 seconds (GLM-5.3-Flash on 2 Sparks, 77% vs 67%); 3.3 times sooner with the same score (Qwen3.8-Flash-Next on 1 Spark); 65% fewer thinking tokens on knowledge and reasoning (GLM-5.3-Flash at 2.05 bpw, 47 vs 48 correct).

1. GLM-5.3-Flash on two DGX Sparks: time budget

Same two Sparks, same speed per token, max effort, one request at a time, wall clock:

Fig. 1: Questions answered correctly within a time limitGLM-5.3-Flash AZ-Focus AZ-Focus ahead
60%70%80%90%100%15 s30 s1 min5 mintime limit (wall clock, two Sparks)67%73%77%83%77%80%83%87%
60 questions (30 MMLU-Pro, 30 BIG-Bench Hard), max effort, one request at a time, same speed per token for both. 4-bit EXL3 on TensorFold with drafts.

Within 15 seconds, AZ-Focus has answered 10 points more questions correctly. The gap narrows as the budget grows but doesn't close within five minutes: the original keeps a tail of questions it is still re-checking. Details, speed tables and receipts are in the two-Sparks post.

2. More Sparks vs fewer tokens

The usual way to get answers sooner is more hardware. It works, but it scales slowly: on Mia's published numbers a second Spark makes Qwen3.8-Flash-Next decode about 1.4× faster (answers in about 72% of the time), and doubling GLM-5.3-Flash from two Sparks to four makes it about 1.6× faster (about 63% of the time). Cutting the tokens instead gets you further on the hardware you already have:

Fig. 2: More Sparks vs fewer tokens: minutes to finish the same 60 questionsOriginal AZ-Focus estimated
GLM-5.3-Flash (4-bit)010203040minutes2 Sparks$13,9003 Sparks$20,8504 Sparks$27,80031.8 min≈24.8 min≈20.2 min15.1 min≈11.8 min≈9.6 minGLM-5.3-FlashAZ-FocusQwen3.8-Flash-Next (INT4)010203040506070minutes1 Spark$6,9502 Sparks$13,90065.7 min≈47.4 min19.7 min≈14.2 minBase modelAZ-Focus
Measured (solid): our time-to-complete runs, 60 questions (30 MMLU-Pro, 30 BIG-Bench Hard), one request at a time, one sample each; scores about the same (Qwen 50/60 each; GLM 50 vs 52). Estimated (hollow): measured time scaled by Mia's single-request decode speeds from mia-ai.net (Qwen INT4 64.4 → 89.3 tok/s on one → two Sparks; GLM-5.3-Flash 60.4 / 77.6 / 95.2 tok/s on two / three / four Sparks). The dashed line carries AZ-Focus's time on the smallest setup across. Spark price: $6,950 each (October 2026).

Doubling the hardware buys you about a third off the wait; AZ-Focus buys you about half (GLM) to two-thirds (Qwen) off, at no extra cost. Or spend the same time and get more questions answered correctly, which is what the budget charts in this post show. On GLM, AZ-Focus on two Sparks finishes before the original does on four, and the two stack: AZ-Focus on four Sparks is faster again.

Other machines sit in the same range. A Mac Studio with the M5 Ultra (256 GB, $10,799) decodes Qwen3.8-Flash-Next at 54 tokens a second in MacStories' tests, about what one Spark does on Mia's recipe in our runs (54–56 with thinking on). A workstation GPU like the RTX PRO 6000 Blackwell (96 GB) now lists at $16,000 for the card alone, and memory shortages have pushed the Spark up from $3,999 to $6,950 this year. Whatever you run on, halving the tokens has the same effect on time-to-answer as doubling decode speed.

3. Qwen3.8-Flash-Next on one DGX Spark: token budget

Tokens are time here: both models generate at about the same speed on the Spark. All 1,298 benchmark answers, and the hardest subset (GPQA Diamond and AIME 2025):

Fig. 3: Questions answered correctly within a token budgetBase model AZ-Focus AZ-Focus ahead
All 1,298 answers20%40%60%80%100%2k4k8k16k32kno limittoken budget (≈ 55 tokens per second on one Spark)64%68%72%78%82%87%70%75%79%83%86%88%Hardest: GPQA + AIME20%40%60%80%100%2k4k8k16k32kno limittoken budget (≈ 55 tokens per second on one Spark)34%41%57%70%91%47%59%71%85%92%
INT4 on Mia's single-Spark TensorFold recipe, default effort (xhigh), 128k output limit. 2k is omitted for the hardest subset (almost nothing finishes).

At about 55 tokens a second, 8k tokens is two and a half minutes. Within that, AZ-Focus has 59% of the hardest questions answered correctly; the base model 41%. With no limit both reach about the same place (1,142 vs 1,135 correct of 1,298). Details in the Qwen post.

4. GLM-5.3-Flash at 2 bits per weight

Lower-bit quantization is the other way to fit a big model on a smaller machine: GLM-5.3-Flash at 2.05 bits per weight is 87 GB, so it fits on one 128 GB Spark. But heavily quantized models tend to second-guess much more, sometimes for tens of thousands of tokens on a question they had already answered correctly (Killy's write-up prompted this test). We built AZ-Focus at turboderp's 2.05 bpw settings and a stock quant with identical settings, and ran both on the same machine:

Fig. 4: Questions answered correctly within a token budget, of 64Stock 2.05 bpw AZ-Focus AZ-Focus ahead
3035404550554k8k16k32k64kno limittoken budget363742455252414346474950
24 MMLU-Pro, 24 BIG-Bench Hard, 10 GPQA Diamond, 6 AIME 2025; one answer each, max effort, 128k output limit; exllamav3 1.4.5 on TabbyAPI. Stock overtakes at 64k because of AIME (below).

On knowledge and reasoning (MMLU-Pro, BIG-Bench Hard, GPQA) AZ-Focus uses 65% less thinking and scores within one answer of stock (47 vs 48 correct of 58). The stock quant's spirals are long: on one MMLU-Pro question it thought for 41,616 tokens where AZ-Focus reached the same correct answer in 396, and on one GPQA question it ran to the 128k limit without answering. Up to a 32k budget, AZ-Focus answers more questions correctly.

Fig. 5: Thinking per answer by benchmark, 2.05 bpwStock 2.05 bpw AZ-Focus
MMLU-Pro24 questions
8,793 tokens · 21 correct
2,351 · 20 correct−73%
BIG-Bench Hard24 tasks
1,694 tokens · 19 correct
349 · 19 correct−79%
GPQA Diamond10 questions
30.8k tokens · 8 correct
13.1k · 8 correct−58%
AIME 20256 problems
53.6k tokens · 4 correct
54.8k · 3 correct+2%
Each row is scaled to the stock quant's thinking. Labels: mean thinking tokens per answer · correct answers. Same questions, same server.

Competition maths is the exception at this bitrate. On AIME both quants think at length (about 54k tokens a problem), AZ-Focus solved 3 of 6 to stock's 4, and one problem ran to the limit on both. That is why stock pulls ahead once the budget reaches 64k. At 4 bits AZ-Focus is ahead on AIME too, so at 2 bits we'd use it for knowledge, reasoning and agent work, and keep a higher-bit model for olympiad maths. Model and full table: GLM-5.3-Flash-AZ-Focus-EXL3-2.05bpw.

Why this happens

Reasoning models made test-time compute the new scaling knob: let the model think longer and it answers more questions correctly. Up to a point. Recent research finds the returns fall off quickly, and that some of the extra thinking actively hurts:

  • Diminishing returns, then regressions. Zhou et al., When More Thinking Hurts (April 2026), find that "marginal returns diminish substantially at higher budgets" and that "extended reasoning is associated with abandoning previously correct answers".
  • Quantization makes it worse. Lotfi et al., Quantized Reasoning Models Think They Need to Think Longer, but They Do Not (May 2026), show that aggressive quantization lowers accuracy and lengthens chain-of-thought: in up to 52% of failures the quantized model reached the correct answer mid-reasoning and then didn't commit to it. The damage concentrates on hesitation tokens ("Wait", "But", "Alternatively"), which is what our 2-bit test shows too.
  • Approaches so far. Lotfi et al. penalise hesitation tokens at decode time (12–23% shorter chain-of-thought). Steering methods act on the model's internal state while it runs: the Halt Vector (August 2026, about a quarter less thinking at held accuracy) and Hyperbolic Entropy Steering (October 2026). Training does the most: Fireworks' Ember-1 (September 2026) reached about 40% fewer tokens on Kimi K3 after more than 50 training experiments. The survey Don't Overthink It covers the field.

AZ-Focus is a weights-only change: no runtime patch, prompt or sampler setting. It cuts thinking by 39–65% depending on model and setting. Every release is tested on agentic coding (SWE-bench Verified) before it ships: level with the original on DeepSeek and Qwen, and about 3 points below on GLM, which we state on its model page.

What this means if you're choosing hardware

  • Measure time to a correct answer, not tokens per second. Two machines with the same decode speed can be minutes apart per question once one of them is running a model that second-guesses.
  • A model change can be worth a hardware tier. Doubling the Sparks takes about a third off the wait; AZ-Focus took half to two-thirds off on the same machines.
  • Low-bit quants are a memory win and a thinking cost. If you run a 2–3 bit build to fit a bigger model, set a token budget and prefer builds that don't spiral.

What this does and doesn't show

  • The gain is largest when the budget is tight and on the questions that make models hesitate. With unlimited time, accuracy is about level (within one or two answers either way on these sets).
  • Same hardware and software for each pair; only the weights differ. Speed per token matched where we measured it (GLM 4-bit and Qwen INT4).
  • The more-Sparks times are estimates, scaled from Mia's published decode speeds; we didn't rerun the 60 questions on two or four Sparks for this post.
  • Agentic coding: level on DeepSeek and Qwen, about 3 points below on GLM; the 2-bit build hasn't had an agentic test yet.
  • Not every setting benefits: at low reasoning effort Qwen3.8 already thinks briefly and AZ-Focus saves almost nothing; at 2 bits, competition maths isn't helped.
  • We haven't run our models on a Mac yet. The BF16 releases are the starting point for MLX conversions; we'd expect the same token savings, but haven't measured them.

Try them

References

  1. S. Zhou et al., When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling, arXiv:2604.10739, April 2026.
  2. S. Lotfi et al., Quantized Reasoning Models Think They Need to Think Longer, but They Do Not, arXiv:2606.00206, May 2026.
  3. D. Jayabahu et al., The Halt Vector: Internalizing a Causal Steering Intervention for Efficient Reasoning, arXiv:2608.28859, August 2026.
  4. Z. Zhang et al., Hesitation Has a Geometry: Entropy-Trained Hyperbolic Probes for Sparse Activation Steering, arXiv:2610.02391, October 2026.
  5. Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models, arXiv:2508.02120.
  6. Fireworks AI, Ember-1, September 2026.
  7. Mia's AI Lab, DGX Spark recipes and benchmarks (GLM-5.3-Flash on 2–4 Sparks; Qwen3.8-Flash-Next on 1–2 Sparks).