CompareToolsMethod @niklaslenz_ai
Local AI · measured at home

I run it at home
so you don't
have to guess.

Which local AI should you run on your hardware? I test setups on my own machines with the same tests. Every number links to the raw output and the exact command.

Find your setup ↓
56complete runs
6memory classes with a run
3,485raw files public
6memory classes

Not up to date yetThe newest models are not on the site yet. I'm testing them now.

My test bench at home: a rack with a Mac Studio and four GB10 boxes between two GPU towers.
Photo: my setup. Background made with AI.
Your hardware

How much memory do you have for AI?

Pick your memory first. A class shows a run only when its speed, quality and memory tests are done.

Last measured Nov 15, 2026

16 GB 1× RTX 5060 Ti6 complete runsAlsoimage 63.4 tok/s →
32 GB 1× RTX 5090 or 2× RTX 5060 Ti14 complete runsAlsoimage 404.4 tok/s →
128 GB 1× GB10 box13 complete runsAlsoimagevideo 141.9 tok/s →
256 GB 2× GB10 boxes8 complete runs 70.6 tok/s →
256 GBMac 1× Mac Studio11 complete runsAlsoimage 306.9 tok/s →
512 GB 4× GB10 boxes4 complete runs 81.8 tok/s →
Newest runs

Latest results

The 6 most recently measured complete runs: speed with one request in tokens per second and the mean of the seven Quality v2 tests. Each tile opens the recipe page with every raw file and the exact command. Pick your memory class.

256 GB MacQwen3.8-Flash-Next1× Mac Studio M5 Ultra · MLX 6-bit, MTP drafts, TensorFold 1.2.8 (Ash recipe), window 262,144 tokens132.6 tok/s · 67.2 % mean of 7 testsMeasured Nov 15, 202632 GBQwen3.6-35B-A3B1× RTX 5090 · NVFP4 (NVIDIA checkpoint) + DFlash2 drafts, TensorFold 1.2.6 (Ash recipe), 131,072-token window404.4 tok/s · 65.2 % mean of 7 testsMeasured Nov 11, 202632 GBQwen3.8-27B1× RTX 5090 · MLX 4-bit (TensorFold files), DFlash2 drafts, TensorFold 1.2.6 (Bob recipe), window 262,144 tokens275.5 tok/s · 64.0 % mean of 7 testsMeasured Nov 11, 2026256 GB MacQwen3.8-Flash-Next (oMLX 0.7.3)1× Mac Studio M5 Ultra · oQ4e-mtp weights (Jundot, revision 2322c888), oMLX 0.7.3 (Jun Kim), MTP depth 3146.5 tok/s · 66.6 % mean of 7 testsMeasured Nov 10 to Nov 11, 202616 GBQwen3.8-27B1× RTX 5060 Ti · EXL3 2.50 bpw, ExLlamaV3 1.6.0 (Mia kit), MTP53.0 tok/s · 60.8 % mean of 7 testsMeasured Oct 19 to Nov 11, 202616 GBQwen3.8-35B-A3B-Distill1× RTX 5060 Ti · Q4_K_M GGUF, llama.cpp b12223 (official release), MoE experts of 27 layers in system RAM63.4 tok/s · 60.3 % mean of 7 testsMeasured Nov 9 to Nov 11, 2026
Extra test

Gauntlet races

Each setup can also play one solo race in the Gauntlet biathlon game: one attempt, no help. The finish times have their own ranking on the game page and a badge in the class tables; they do not change the order of the tables or whether a run is shown. All races and the Gauntlet ranking.

16 GBQwen3.8-27B TQ_XSRTX 5060 Ti 16 GB283.5 s finish time · 7 / 20 targets · rank 4 of 9Thinking off16 GBQwen3.8-27B EXL3 2.5bpwRTX 5060 Ti 16 GB278.2 s finish time · 8 / 20 targets · rank 3 of 9Thinking off128 GBOrnith 1.5 122B1 DGX Spark 128 GB317.5 s finish time · 5 / 20 targets · rank 6 of 8Thinking off128 GBOrnith 1.5 122B1 DGX Spark 128 GB377.7 s finish time · 3 / 20 targets · rank 8 of 8Thinking off256 GBGLM-5.3-Flash2 DGX Sparks 256 GB352.9 s finish time · 3 / 20 targets · rank 7 of 8Thinking off128 GBThinkingCap Qwen3.8-27B Kearuga1 DGX Spark 128 GB202.9 s finish time · 13 / 20 targets · rank 1 of 8Thinking off
How I test

Five tests. No vibes.

Speed v2
tok/s

How fast the answer appears: tokens per second with one request, and the time to the first token at 2K, 8K and 32K.

Quality v2
7 tests

Fixed question lists with a margin of error next to each number: MMLU-Pro, GPQA, IFEval, LiveCodeBench, long text and GPQA with thinking.

Memory test
context

The largest context that loads and answers, the memory it needs and the decode speed there.

Game test
Gauntlet

The model plays one solo race in a biathlon game: four tasks, twenty shots, penalty loops. An extra test with its own ranking and a badge; a run counts without it.

sparkDash
Mia's tool

Mia's own benchmark tool, run unchanged, in its own column. Never mixed with my numbers.

Checked, not assumed

  • A run is in the class tables only when every part is done: Speed v2, all seven Quality v2 tests and the memory test; sparkDash is added where the setup ran it. The game result is added as soon as the setup has played its race. Every number links to its raw file. The chip “independent check passed” marks a page whose numbers an independent check compared with the raw files.
  • The exact software version is pinned for every run.
  • If an answer gets cut off by the length limit, it counts as wrong. The setup page says how many were cut off.
  • The tables are ordered by speed (tokens per second) and quality. A run counts with or without a Gauntlet race, and there is no overall score, only the raw numbers. The Gauntlet is an extra test with its own ranking on the game page and a badge in the tables. How it is made.