The documented NVFP4 checkpoint is about 126 GiB. It cannot sit entirely in a 32 GB GPU. A 128 GB Mac is not a verified configuration, and “6B active” does not mean a 6B download. This is a source-checked hardware guide; I have not benchmarked Flash-Next myself.
What changed: a new architecture preview and an NVFP4 release
Qwen describes Flash-Next as an experimental architecture preview for Qwen4. The model combines a 125B main network, a 51B n-gram embedding component and a 4B multi-token prediction component. About 6B parameters are active per token. Those are different quantities, with different consequences for a machine you can actually run. Qwen’s model card.
NVIDIA’s NVFP4 export, released on 31 August 2026, makes the hardware question more interesting. It uses mixed precision: routed experts use NVFP4, while other components retain BF16 or FP8. Calling the whole package “4-bit” hides that distinction. NVIDIA’s checkpoint and precision breakdown.
Why “6B active” does not mean it fits like a 6B model
Active parameters describe how much of a mixture-of-experts model participates in a token’s computation. Inactive experts still need to be stored somewhere and made available when selected. Offloading can change where they live, but it does not make their bytes disappear.
A useful first calculation is parameters × bits ÷ 8. Applying four bits uniformly to 180 billion parameters gives 90 billion bytes: 90 GB, or about 83.8 GiB. That is an idealized calculation, not the size of the published mixed-precision checkpoint. It also leaves out runtime buffers, the KV cache and operating-system memory.
| Format | Weight size | What the figure means |
|---|---|---|
| Hypothetical uniform 4-bit | 90 GB / 83.8 GiB | Arithmetic only; not a downloadable configuration verified here. |
| Documented NVFP4 | About 126 GiB | SGLang’s mixed-precision checkpoint recipe. |
| Documented FP8 | 172.78 GiB | vLLM recipe weight size, before serving headroom. |
| Documented BF16 | 335.28 GiB | vLLM recipe weight size, before serving headroom. |
The NVFP4 total includes about 78 GiB of expert and dense weights and a 47.7 GiB FP8 n-gram table. GB and GiB are not interchangeable: 126 GiB is approximately 135.3 GB. SGLang’s memory breakdown; vLLM’s checkpoint sizes.
RTX 5090, Mac, DGX Spark: what can the evidence support?
Read this table as a capacity and support check. A “does not fit entirely” result leaves open future quantizations or offload experiments; it is not a claim that every possible implementation is impossible.
| Hardware tier | Assessment | Evidence boundary |
|---|---|---|
| RTX 5090 / 32 GB VRAM | The documented NVFP4 weights do not fit entirely in VRAM. | No single-5090 working recipe or speed measured here. |
| 64 GB Mac | Insufficient memory for the documented resident checkpoint. | CPU and GPU share the same unified memory pool. |
| 128 GB Mac | The documented checkpoint exceeds the nominal memory budget. | No compatible Mac offload recipe verified in this guide. |
| 192 GB Mac | More raw capacity; not proof of a working setup. | Architecture support, quantization, allocation limits and speed still need testing. |
| RTX PRO 6000 Blackwell / 96 GB | SGLang documents an embedding-table CPU-offload recipe. | Requires at least 64 GB of free host RAM, plus checkpoint page-cache headroom. |
| One DGX Spark / 128 GB | SGLang documents file-backed NVMe embedding offload. | Not an entirely memory-resident configuration. |
The workstation and Spark configurations above come from the SGLang cookbook. Moving data between CPU and GPU on unified-memory hardware does not create a second independent memory pool. Likewise, a Linux/CUDA recipe is not evidence that an MLX implementation exists.
The exact runtime matters as much as the memory
Do not begin with a generic install command and assume the architecture is supported. Record the checkpoint revision, runtime build, quantization and launch flags together. These are the source-documented paths checked on 9 September 2026:
- SGLang: the local NVFP4 recipes specify
lmsysorg/sglang:dev-qwen38-next-localand theqwen4-main-squashedbranch at commit4ccff141db. Follow the matching hardware recipe. Build and launch details. - vLLM: its recipe specifies the dedicated
vllm/vllm-openai:qwen38-flash-nextimage and says PyPI installation is not supported for that recipe. Its published FP8 paths target much larger GPU systems. Supported configurations. - NVIDIA’s NVFP4 export: the model card identifies vLLM commit
d4d703caf908786416585ceb1f369e2e0363358bor later for support without MTP, with an additional change required for MTP at the time of its instructions. Recheck the card before building. Compatibility notes.
What I would verify before calling a setup usable
- Loads: record peak host RAM, VRAM and disk requirements while loading, not just the final idle allocation.
- Generates: run a short prompt and verify coherent output, tokenizer behavior and the intended precision.
- Handles the actual task: test your real prompt lengths, tool calls and concurrent requests. Record time to first token separately from decode throughput.
- Survives pressure: watch swapping, allocation failures and latency after repeated requests. Leave room for other applications.
Qwen lists a native context of 262,144 tokens, extensible to one million. That is an architecture capability, not a promise that a particular desktop can serve it. The vLLM recipe explicitly notes that a single 262K-token request was not tested. Begin with a modest context and increase it only with measurements. Context specification; Test limitation.
A useful experiment before buying more hardware
Choose a model that already fits your machine and establish a repeatable baseline. My 27B Mac guide, same-model Mac versus RTX 5090 comparison, and RTX 5090 serving recipes cover that smaller, measurable step. Flash-Next becomes a worthwhile next experiment when its runtime path and memory budget answer a specific need.
Update policy: this guide will change when a reproducible smaller-GPU or Apple Silicon recipe is verified. Until then, capacity estimates remain estimates, and vendor recipes remain attributed evidence.