Qwen3.8-Flash-Next System Requirements: VRAM, RTX 5090 & Mac

Qwen3.8-Flash-Next is a mixture-of-experts model that Qwen describes as an experimental architecture preview for Qwen4. It combines a 125B main network, a 51B n-gram embedding table and a 4B multi-token prediction component. About 6B parameters are active per token, but the whole model still has to be stored in memory or on disk. Qwen’s model card.

Check if a model fits →Mac guide →RTX 5090 guide →

Requirements at a glance

The table separates reported configurations from an unverified memory estimate. It does not establish the minimum hardware for every checkpoint and context length. Official means Qwen, NVIDIA, the SGLang, vLLM or llama.cpp projects, or Ollama; community means third-party quantizers, forums and blogs. Speeds are single-user generation speeds as reported, mostly on late-August builds. The engines have gained kernels and MTP support since, so treat them as starting points.

Minimum and recommended

• Reported consumer setups: a 24–32 GB GPU paired with 96 GB of system RAM, or a 64 GB Mac reading the n-gram table from SSD. These reports do not establish a universal RAM minimum or predict your speed. Low-bit builds also need quality checks for your task.
• Recommended for daily local use: a 128 GB Mac at 4-bit, or a DGX Spark or RTX PRO 6000 with the official SGLang recipes. On a Mac, the vendor guides raise macOS’s GPU memory limit (iogpu.wired_limit_mb) before loading a 100 GB build.
• Higher-precision formats: the FP8 weights alone are 172.78 GiB and BF16 335.28 GiB. At those sizes it is a job for a multi-GPU server.

These picks are my reading of the sources linked below. The RTX 5090 report, 64 GB Mac card and SGLang cookbook were rechecked on 31 October 2026; other figures retain their 2 October source check. None is my own measurement.

Qwen3.8-Flash-Next on an RTX 5090

The cited RTX 5090 report pairs the 32 GB card with 96 GB of system RAM and 1–2-bit GGUFs of 72–79 GB. It reports about 52 tok/s at 1-bit and 44 at 2-bit. That is a community benchmark rather than my own measurement, and 4-bit is untested in it.

NVIDIA’s mixed-precision export and the releases that now support it

NVIDIA’s NVFP4 export, released on 31 August 2026, uses mixed precision. Routed experts use NVFP4; attention, shared experts and other layers stay in BF16; and the n-gram table is FP8. Calling the whole package “4-bit” hides that mix. NVIDIA’s checkpoint and precision breakdown.

The 2 October review found support in vLLM 0.29.0 and 0.30.0, llama.cpp v0.5.0, Ollama and LM Studio. The SGLang cookbook checked on 31 October requires 0.5.22 or later. Runtime details below distinguish the current SGLang requirement from the earlier checks.

Why “6B active” still needs memory for the whole checkpoint

Active parameters describe how much of a mixture-of-experts model participates in a token’s computation. Inactive experts still need to be stored somewhere and made available when selected. Offloading moves them to system RAM or SSD; they still take up space there.

A first estimate is parameters × bits ÷ 8. Applying four bits uniformly to 180 billion parameters gives 90 billion bytes: 90 GB, or about 83.8 GiB. That arithmetic is not the size of a specific checkpoint. The estimate also leaves out runtime buffers, the KV cache and operating-system memory.

To add those for your own machine, model and context, try the Will this model fit? calculator; it keeps the n-gram table separate, as below.

The NVFP4 total includes about 78 GiB of expert and dense weights and a 47.7 GiB FP8 n-gram table. GB and GiB are not interchangeable: 126 GiB is approximately 135.3 GB. SGLang’s memory breakdown; vLLM’s checkpoint sizes; a community GGUF file list.

The runtime looks rows up in the n-gram table instead of computing through it, so the table can live in slower memory. Every small setup in the first table relies on this. The official recipes keep it in host RAM (RTX PRO 6000) or in a file on NVMe (DGX Spark). Several community builds page it from SSD on 64 GB Macs. On unified-memory hardware, moving data between CPU and GPU does not create a second memory pool; it only changes what has to stay resident.

Runtime versions that support Flash-Next

Check that your runtime build supports this architecture before you install it, and record the checkpoint revision, runtime build, quantization and launch flags together. The SGLang requirement below was rechecked on 31 October 2026. Other runtime notes describe the documentation checked on 2 October; check the linked release notes before installing:

• SGLang: 0.5.22 or later, according to the cookbook. Its verified DGX Spark and RTX PRO 6000 numbers were measured on the earlier lmsysorg/sglang:dev-qwen38-next-local image at commit 4ccff141db, so follow the matching hardware recipe. On a Spark the first boot builds the n-gram file in about 10 minutes, and the recipe deletes that file before each boot because reusing it is slower. Build and launch details.
• vLLM: 0.29.0 added the model. NVIDIA’s NVFP4 export needs 0.30.0, which contains both the commit its card names and the MTP fix the card still lists as pending. The recipe now recommends the nightly Docker image, and it still notes that a single 262K-token request was not tested. Supported configurations; vLLM releases.
• llama.cpp: supported since 27 August and included in the v0.5.0 release. MTP reached the main branch on 1 October, after v0.5.0. Most of the consumer setups above run on it, with GGUF files. Model support; MTP.
• Apple Silicon: At the 2 October check, Apple’s mlx-lm support was still in a pull request. The mlx-vlm library has supported the model since 26 August, with MTP since 7 September, and Ollama’s MLX build runs it. mlx-lm status; mlx-vlm releases.
• Ollama and LM Studio: Ollama lists the 105 GB MLX build and a 120 GB 4-bit GGUF, and LM Studio’s changelog lists support. Ollama builds; LM Studio changelog.
• NVIDIA’s NVFP4 export: the card names vLLM commit d4d703caf908786416585ceb1f369e2e0363358b or later for support without MTP. That commit and its MTP prerequisite both shipped in vLLM 0.30.0. Compatibility notes.

Four checks before calling a setup usable

1. Loads: record peak host RAM, VRAM and disk requirements during the load, as well as the final idle allocation.
2. Generates: run a short prompt and verify coherent output, tokenizer behaviour and the intended precision.
3. Handles the actual task: test your own prompt lengths, tool calls and concurrent requests. Record time to first token separately from decode throughput.
4. Survives pressure: watch swapping, allocation failures and latency after repeated requests. Leave room for other applications.

Qwen lists a native context of 262,144 tokens, extensible to one million. That is the architecture’s limit; the vLLM recipe notes that a single 262K-token request was not tested. Start with a modest context and raise it only after measuring. Context specification; Test limitation.

What to test before buying more hardware

Choose a model that already fits your machine and establish a repeatable baseline. My 27B Mac guide, same-model Mac versus RTX 5090 comparison, and RTX 5090 serving recipes cover that smaller, measurable step. Flash-Next is worth trying once its runtime path and memory budget meet a specific need.

Update scope: The 31 October correction updates the SGLang minimum version, the RTX 5090 and 64 GB Mac descriptions, and the distinction between measured setups and memory estimates. It does not add a first-hand benchmark.

Common questions

Sources

• Official: Qwen’s model card, NVIDIA’s NVFP4 card, the SGLang cookbook, the vLLM recipe, llama.cpp’s MTP pull request with DGX Spark timings, Ollama’s builds.
• Community: an RTX 4090 llama.cpp setup, an RTX 5090 offload benchmark, M4 Max MLX and llama.cpp timings, a 64 GB Mac build, an M2 Ultra benchmark.

Suhail Mohebi · published 9 September 2026 · the cover is a generated illustration