Will a local LLM fit?
Start with your available memory, then allow room for context and the runtime. Use the calculator before downloading a larger model.
Estimate memory →Local AI: hardware, models and measured tests
My M4 Max and RTX 5090 tests, plus credited research on other workstations: which local AI models fit which machine, how fast they run and what each comparison can establish.
Start with the decision
Choose a model, a runner or a test by the job you need to finish.
LM Studio or Ollama?
Compare the setup and local API with the same model file. The test record separates successful requests from claims about speed or model quality.
Compare the runners →Which model can do the task?
Use the coding comparison to inspect the task pack, scored attempts and failures. A fast answer still needs to pass the job's checks.
Read the coding tests →Local AI means running models locally: inference happens on your own hardware rather than through a cloud-based AI provider. A runner is the application that loads and serves the model; changing the runner does not make the model a different size or remove its licence conditions. Open weights and open source describe different things, so check the model's own licence before using or redistributing it.
For a UAE team, test the language and documents the workflow actually uses. Include Arabic cases when they are part of the work. Check where prompts, logs and connected tools send data; a local model alone does not establish that a whole workflow stays on the device. Use the private AI data-flow checklist to map those destinations, then the workflow-readiness guide to assess the task.
Pages labelled Test report runs on hardware I own: a MacBook Pro with an M4 Max and 128 GB of unified memory, and a Windows workstation with one RTX 5090 (32 GB) that I reach over Tailscale. Each measured number carries the machine and the date it was taken on, because a local-model result is only as good as the runtime, the quantization and the context it was measured with. The research reviews quote published figures instead and credit their testers, including the five-part series on workstation benchmarks, AMD configurations and local AI costs.
Start with the first question: will the model fit? The calculator adds up weights, KV cache and overhead for a GPU or a Mac. Model weights are only part of the memory budget. Quantization, context length and the KV cache also affect what fits; check the model and runtime you intend to use.
Then the engines. On the RTX 5090, SGLang with a drafter decoded Qwen3.8-27B at 139.7 tok/s against vLLM's 100.8 with MTP and 84.5 without speculation, at the same 32K context. On the M4 Max, a DFlash 2 drafter took the same model from 16.7 to 47.2 tok/s; native MTP added about 24%. The same 27B read prompts 46× faster on the RTX 5090 than on the Mac and decoded 5× faster, but the 32 GB card caps the context window and the 128 GB Mac holds what the card must page around.
The performance tests record the runtime, quantization and context, with scripts and raw runs in the public lab repository ↗. The LM Studio/Ollama comparison checks response format instead and includes its own downloadable record. Read each test’s method before comparing its numbers.
Will it fit?
The calculator I open first, and the three numbers it rests on.
LLM VRAM calculator: will this model fit your GPU or Mac?
My calculator adds up weights, KV cache and overhead to show whether Qwen 3.8, gpt-oss or Gemma 4 fits your GPU or Mac, with every formula shown.
- 32 GB
- of Mac memory runs a 27B model at 4-bit, about 16.1 GB of weights
- 96–128 GB
- runs the 8-bit build (29.5 GB) with room for context
- 2×
- the context that fits with an FP8 KV cache, at negligible loss
Local AI on a Mac
The M4 Max with 128 GB: which Mac, which quantization, and what speculation changes.
5 entries
Local LLMs on Apple Silicon: which Mac, RAM and speed
A 32 GB Mac runs a 27B model at 4-bit; 96–128 GB runs 8-bit with room for context. Measured M4 Max speeds, KV-cache costs and when a GPU box wins.
Qwen3.8-27B on M4 Max: 8-bit MLX review and Mac RAM needs
32 GB Macs should start with 4-bit (about 16.1 GB); 96–128 GB Macs have room for 8-bit (29.5 GB). My M4 Max MLX review and the Opus 4.6 comparison.
Speculative decoding on an M4 Max: MTP vs DFlash 2, measured
A DFlash 2 drafter took my Qwen3.8-27B from 16.7 to 47.2 tok/s on a 128 GB M4 Max; native MTP added 24%. Plus the log figure that misled me.
Local LLMs for coding on an M4 Max: Qwen vs Gemma vs Muse
Three NVFP4 local models ran eight coding tasks through Codex CLI on a 128 GB M4 Max, 72 scored attempts. Qwen3.8-27B passed all 24; Gemma 4 was fastest.
LM Studio vs Ollama on a Mac: setup and a same-model test
LM Studio and Ollama ran the same GGUF on a 128 GB M4 Max. Setup differences, local APIs and why correct answers still failed strict JSON parsing.
The RTX 5090 workstation
One 32 GB card, set up and measured remotely: the plan, the serving configs, the engine comparison and the Mac side by side.
6 entries
Building a home AI lab on an RTX 5090: the plan
Why I moved part of my AI experiments from metered cloud GPUs to one RTX 5090 workstation, what I planned to test, and where the measured build ended up.
Remote RTX 5090 setup: Codex, Qwen3.8-27B and Tailscale
Codex installed Qwen3.8-27B on my RTX 5090 from my Mac, measured four vLLM and SGLang recipes, then moved access onto Tailscale with no public port open.
Qwen3.8-27B on RTX 5090: SGLang vs vLLM Serving Configs
Four tested configs for Qwen3.8-27B on one 32 GB RTX 5090, from 84.5 tok/s (vLLM baseline) to 139.7 (SGLang with a drafter), and the two tuning traps.
SGLang vs vLLM on an RTX 5090: speed, memory and context
I ran one 27B model on an RTX 5090 at 32K context: SGLang with a drafter hit 139.7 tok/s, vLLM 100.8 with MTP. Prompt speed, memory and which to pick.
M4 Max vs RTX 5090 for local LLMs: same Qwen 3.8, measured
On the same 27B model an RTX 5090 read prompts 46× faster than a 128 GB M4 Max and decoded 5× faster, but only 3–4× faster with speculative decoding.
Tool calling at 250K context: a 40-check test on a local 35B
I set a rule at the top of a 262K window on a local 35B model. It held on every tool call 250K tokens later, where the suite scored 39 of 40 checks.
Reading local AI benchmarks
Published workstation and GPU tests, with credits, configuration details and limits.
5 entries
Mac Studio and RTX cards: reading published LLM tests
Published Mac Studio, RTX 5090 and RTX PRO 6000 tests separate prompt speed from memory capacity. What each comparison can establish.
DGX Spark 64GB: what published tests can tell us
NVIDIA’s 64GB DGX Spark announcement, older 128GB tests and the independent review still missing. Vendor claims are separated from measurements.
AMD local LLM tests: the configuration behind the score
Credited Radeon R9700 and Strix Halo results, corrected Puget measurements and why runtime, concurrency and speculative decoding belong beside the score.
Why local LLM benchmark charts disagree
Cache state, concurrency, tokenizers and model files explain conflicting local LLM scores. Compare published tests without hiding their limits.
Local AI cost calculations: power, usage and payback
A method for estimating local AI costs using credited power measurements and explicit assumptions about utilization, avoided spending and running costs.
Choosing and fitting models
Memory maths, quantization, the KV cache, model reviews and the running test log.
8 entries
LLM VRAM calculator: will this model fit your GPU or Mac?
My calculator adds up weights, KV cache and overhead to show whether Qwen 3.8, gpt-oss or Gemma 4 fits your GPU or Mac, with every formula shown.
Qwen3.8-Flash-Next System Requirements: VRAM, RTX 5090 & Mac
Qwen3.8-Flash-Next hardware reports for RTX 5090 and Macs: checkpoint sizes, RAM and SSD offload, runtime requirements, and measurement limits.
LLM quantization: how to choose Q4, Q8 or NVFP4 locally
The memory maths for Q4, Q8 and NVFP4 on a 32 GB RTX 5090, where quality starts to drop, and the checks I run before trusting a quantized model.
KV cache quantization for local LLMs: FP8 vs FP4
I run an FP8 KV cache: it doubles context at negligible loss. A 4-bit cache halved reasoning scores in SGLang’s own table. The maths for a 32 GB card.
Popular local LLMs vs abliterated versions: what fits 128 GB
The most downloaded Hugging Face models that fit a 128 GB M4 Max, M5 Max or DGX Spark, and what changes when a model is abliterated.
Uncensored local models: what they are and when I use one
Why some of my work runs on open-weight models on my own laptop: privacy, cost and fewer refusals on legitimate work. What uncensored and abliterated mean.
Local LLM field notes: on-device, private and open weights
On-device, private and open-weight are three different promises. My running local-model test log, starting with oMLX serving two 4-bit Qwen3.8-27B builds.
DeepSeek V4.1 Flash 1017: benchmarks and pricing
DeepSeek’s 17 October update reports 77.0 on DeepSWE and 42.3 on Terminal-Bench 4.0 with shorter outputs. API prices, Search billing and how I’d test it.
How I test
The same three things sit behind every number on these pages.
The running test log
Runner, quantization, configured context, latency, failure modes and cloud comparisons, appended as each model is tried.
Local LLM field notes →The test contract
Record a model's full identity and measure prefill, decode and real jobs apart, so two results can be compared.
Get the template →The scripts and raw runs
Launch scripts, the benchmark harness and raw runs, in a public repository that six of these notes cite.
local-llm-lab on GitHub ↗