Local LLMs for coding on an M4 Max: Qwen vs Gemma vs Muse
Three NVFP4 models ran the same eight TypeScript tasks on my 128 GB M4 Max, with Codex CLI driving and hidden tests plus a blind review grading. Qwen3.8-27B passed all 24 attempts, Gemma 4 was fastest, Muse ran out of time on refactors.
Short answer
On this cohort and this workload, Qwen3.8-27B was the model that finished the job every time. Over eight small TypeScript tasks, three attempts each, driven by Codex CLI with hidden acceptance tests and a blind patch review, Qwen passed 24 of 24 attempts. Gemma 4 26B A4B passed 21 of 24 after review and was about a third faster per attempt. Muse Glimmer 30B passed 21 of 24 but spent six attempts at the ten-minute cap and never solved the one refactor that separated the three.
Results ↓Test record and downloads ↓Local AI index →
“Best” here means best out of three NVFP4 builds, on a MacBook Pro M4 Max with 128 GB of unified memory, through Ollama 0.40.13’s MLX engine, on a synthetic repository of a few hundred lines. The Apple Silicon guide covers which Mac and how much memory; this note is about what three builds do once they fit. If your work is bigger than that, this is a starting point rather than a verdict.
The three builds, as pulled from the Ollama library on 30 October 2026:
| Config | Ollama tag (digest) | Publisher model | Architecture | Download |
|---|---|---|---|---|
| Q | qwen3.8:27b-nvfp4 (5642e97495e1) | Qwen/Qwen3.8-27B | dense, 27.8B | 18 GB |
| G | gemma4:26b-nvfp4 (f60799545325) | google/gemma-4-26B-A4B-it | mixture of experts, 26.2B total | 16 GB |
| M | muse-glimmer:30b-nvfp4 (6a78d632287f) | meta-models/Muse-Glimmer-30B | 29.8B | 17 GB |
Each was wrapped in a Modelfile that set a 64K context window and nothing else.
Results
| Configuration | Passed / 24 | Tasks solved / 8 | Elapsed, passing (median, range) | Failures / timeouts | Peak model in memory |
|---|---|---|---|---|---|
| Q · Qwen3.8-27B | 24 (22 finished, 2 at the cap) | 8 | 213 s (132–601) | 0 / 2 | 27.9 GB |
| G · Gemma 4 26B A4B | 21 (22 by the tests, 1 removed in review) | 8 | 139 s (64–339) | 2 / 0 | 23.3 GB |
| M · Muse Glimmer 30B | 21 (18 finished, 3 at the cap) | 7 | 384 s (201–602) | 3 / 6 | 20.5 GB |
A passed attempt means all three gates held: the repository’s own tests still pass, a hidden acceptance suite the model never saw passes, and a reviewer who did not know which model wrote the patch confirmed it does what the task asked, with no deleted or weakened tests and no unrelated damage. “Tasks solved” counts a task once if any of its three attempts passed all three gates. “At the cap” means the agent was still working when the 600-second limit hit; I scored the workspace as it was left. Those count as passes because the code was complete, but I report them separately because the agent never said it was done.
Elapsed time is for passing attempts only, measured from the start of the Codex process to the end of grading. Failed and timed-out attempts are in the failures column so they do not vanish from the speed story. “Peak resident model” is what ollama ps reported as allocated, which is not the same thing as kernel memory use; the Ollama runner process itself peaked between 15 and 17 GB of resident memory for all three.
Per task, with three attempts each (counts are reviewed passes):
| Task | Q | G | M |
|---|---|---|---|
| B1 Fix the date-range filter | 3 | 3 | 3 |
| B2 Fix pagination | 3 | 3 | 3 |
| R1 Extract duplicated validation | 3 (2 at cap) | 1 | 0 (3 timeouts) |
| R2 Split report aggregation from formatting | 3 | 2 | 3 |
| T1 Fix a test that passes for the wrong reason | 3 | 3 | 3 |
| T2 Isolate leaking test state | 3 | 3 | 3 |
| F1 Add a search filter | 3 | 3 | 3 |
F2 Add --format json to the CLI | 3 | 3 | 3 (3 at cap) |
Six of the eight tasks were solved by every model on every attempt. The whole difference between the three sits in the two refactors and in how long the work took.
What was tested
The repository is a small TypeScript “ledger” app I wrote for this test: CSV and JSON importers, a date filter, pagination, a text report and a CLI, with 17 visible tests. It runs on Node 22 with no dependencies. Each task is a plain-English brief of the kind I would give a colleague: two bug fixes (B1, B2), two refactors (R1, R2), two test repairs (T1, T2) and two small features (F1, F2). The bug tasks start from code with visible tests that pass for the wrong reason; the test-repair tasks start from tests that are green but broken.
The agent was Codex CLI 0.163.0 in exec mode, pointed at Ollama on the same Mac, with its workspace-write sandbox: it could edit only inside the task’s folder and the sandbox had network access switched off. Each attempt got a fresh copy of the repository, a cap of 10 minutes or 40 tool calls, and no help from me. The hidden tests and reference patches sat outside the workspace with their file permissions removed while an attempt ran; they were never mentioned in the prompt.
Every task ran three times per model, 72 scored attempts in all, in a seeded interleaved order so no model got a run of easy tasks while the machine was cool. One model was loaded at a time. Thinking was left at each model’s Ollama default (Qwen medium, Gemma on, Muse high) and Codex sent no reasoning-effort override. Sampling was left at each publisher’s defaults, which differ: Qwen ships top_k 20, Gemma and Muse top_k 64, all at temperature 1. Identical settings would not have been fairer; I note the difference rather than pretend it away.
Grading was mechanical first: the visible tests, then the hidden acceptance suite (between 1 and 21 tests per task), then a few checks for skipped assertions, test-script edits and visible tests that went missing. After the whole matrix finished, every patch was reviewed from an anonymised packet with the model names scrubbed out. The reviewers were fresh agent instances that saw only the task brief, the diff, the final message and the evaluator counts.
I ran the tests with my usual desktop apps open, which matters on a Mac: before every attempt a short probe measured how fast the loaded model processed a fresh prompt, with a floor of 150 tokens per second below which the run would have waited. The floor was never hit, but prompt processing for Qwen and Muse sat at roughly 190 to 290 tokens per second throughout, with the GPU already 80 to 100 percent busy. A clean machine would be faster in absolute terms for all three.
Where each model worked or failed
The refactor that separated them. R1 asks for the duplicated per-row validation in the CSV and JSON importers to move into one shared module, with every error message and its order preserved. It sounds small. It needs two files changed in step, a third created, and a CSV-specific quirk handled (the CSV importer converts the amount before validating; the JSON importer does not).
Qwen solved it three times, but two of those ran to the ten-minute cap: the patches were complete, the tests were green, and the agent was still writing extra tests and re-checking its work when time ran out. Gemma solved it once. On its second attempt it widened the amount check to accept strings, which the hidden tests caught; on its third it created the shared module and then changed neither importer, leaving the new code unused. Muse timed out all three times. Twice it left a near-complete patch that dropped the “entry must be an object” guard, so a null row produced a raw TypeError instead of the required “row n:” message. Once it had migrated only the CSV side when time ran out.
The pass that review took away. On R2, splitting the report into an aggregation step and a formatting step, Gemma’s third attempt passed every hidden test. The reviewer rejected it anyway: the new aggregation sorted categories with localeCompare where the original used JavaScript’s default sort. For lowercase ASCII category names, which is all the tests used, the output is identical. For a category called “Zebra” or “_misc” it differs, and the brief required the text report to stay byte-identical. That is the one case in 72 where the human-style review changed a verdict, and it is why I would not trust test counts alone.
Speed and habits. Gemma finished a typical attempt in a little over two minutes. Its decode rate in the probe was around 135 tokens per second against roughly 55 for Qwen and 28 for Muse (the speculative decoding note measures what a drafter does to Qwen’s rate on this Mac), and a Gemma attempt also read more of the repository, with a median of 13 tool calls and the largest prompt volume of the three. It was also the messiest: 20 times across 24 runs it tried to call Codex’s apply_patch through a bash command with the wrong syntax before recovering, four times it used GNU-style sed -i that fails on macOS, and once it left a stray my_patch.patch in the repository root. The reviewer passed that run on the grounds that the file was harmless; the brief had asked for edits only under src/ and test/, so count it as a judgement call.
Muse’s problem was arithmetic. At roughly 28 tokens per second of output, a multi-file refactor with high reasoning effort does not fit in ten minutes. All three of its F2 attempts hit the cap with a complete, passing patch in the workspace, and its R1 attempts hit the cap without one. In one T2 run Muse emitted a malformed tool-call parameter, Codex logged a stream error and reconnected, and the attempt finished and passed; I count it as completed with the error noted.
Qwen was the tidiest: a median of 10 tool calls, the fewest failed commands, and the only model with zero rejections. It wrote its own tests more often than the other two.
What to choose
The best local LLM for coding in this test was Qwen3.8-27B, within these three builds and this workload. I did not test GPUs with 8 to 32 GB of VRAM here, so use the VRAM calculator to see what fits yours. For small, well-specified coding jobs on a 128 GB M4 Max, with the models as they shipped in the Ollama library on 30 October 2026:
- Qwen3.8-27B was the most consistent model in these eight tasks across three attempts each. Its output still needs review. Budget three to four minutes per task and expect refactors that touch several files to take the full ten.
- Gemma 4 26B A4B if you iterate interactively and read every diff. It is the fastest by a wide margin and solved every task at least once, but one of its 24 patches would have shipped a subtle behaviour change that the tests did not catch, and it fumbles the tool interface more than the others.
- Muse Glimmer 30B only with a longer time budget than mine. Its completed patches were sound, but on this Mac its output rate makes anything beyond a single-file fix a ten-minute wait.
None of this says anything about large repositories, long sessions or other Macs, and a passing patch is where my local-model test sequence starts rather than ends. The earlier notes on whether a model will fit and on tool calling at 250K context cover memory and long-context behaviour; this test was deliberately short-context.
Limitations
- The workload is synthetic and small: eight tasks on a repository of a few hundred lines. Three attempts at one task are three samples of the same task, not three tasks. A 24-versus-21 gap is a real difference on this workload and nothing more.
- These are three different NVFP4 recipes of three different architectures, compared as model-plus-runtime configurations. The quantization guide explains why “4-bit” files are not interchangeable.
- The machine was not idle. GPU contention from desktop apps slowed prompt processing for all three models; the per-run probe values are in the CSV so the effect is visible, but absolute times would be lower on a clean machine, and I did not measure by how much.
- The blind review was done by fresh agent instances reading anonymised packets, dispatched by the same agent that ran the matrix. The packets hid which model wrote each patch; the process was not independent of me.
- The sandbox had network access off and Codex was configured to use only the local Ollama provider. I did not run a fully offline test, so I make no claim that the whole agent works with the network unplugged.
- Temperature 1 with publisher-default sampling, so repeated runs vary; three attempts show the spread, not the mean.
Reproduce it
Test record · 30–31 October 2026
- Machine
- MacBook Pro · Apple M4 Max · 128 GB unified memory · macOS 27.0.1
- Runtime
- Ollama 0.40.13 (MLX engine), one model resident at a time
- Agent
- Codex CLI 0.163.0 in
execmode, workspace-write sandbox, network off; the fixture’s tests ran on Node v22.23.4 - Models
qwen3.8:27b-nvfp4(5642e97495e1) ·gemma4:26b-nvfp4(f60799545325) ·muse-glimmer:30b-nvfp4(6a78d632287f), pulled 30 October 2026, each wrapped in a Modelfile that setsnum_ctx 65536- Protocol
- Version
2026-10-30-v1-frozen: 64K context, 600 s or 40 tool calls per attempt, seed 20261030, three attempts per task per model, hidden tests and reference patches outside the workspace · protocol JSON - Dates
- 30 and 31 October 2026, 72 scored attempts, blind review after the full matrix
Downloads: task pack (fixture, briefs, overlays, Modelfiles and the protocol; 27 KB zip, no hidden tests) · protocol JSON · 72-row results CSV with per-run timing, tokens, memory and probe values. The hidden acceptance tests stay private so the tasks remain usable.
Related notes: Qwen 3.8 on my M4 Max for the memory baseline, the same model on two machines for why the Mac matters, and the local AI index.
