I Set Up My RTX 5090 AI Workstation Without Touching It

Four days earlier, I published the plan for my personal AI lab. The hardware was real, but most of the verbs were still in the future tense: I would install the models, compare the runtimes, test long context and connect the machine to agent workflows.

This is the receipt for what happened next.

I used Codex on my Mac to operate the Windows workstation over the local network. It inspected the machine, built the serving environments, installed and tested Qwen 3.8 27B, created separate vLLM and SGLang recipes, benchmarked them under matched conditions, repaired persistence problems and kept every inference listener on localhost. I then left home and repeated the acceptance test through an encrypted Tailscale route.

§1One machine became the control plane

The first job was not installing a model. It was proving which machine Codex was controlling and how.

A fresh SSH connection verified the Windows host, account boundary, operating system, RTX 5090 and available storage. Key authentication and strict host-key checking were already working. Remote Login on the Mac remained disabled because the required direction was Mac to workstation, not the reverse.

The initial architecture was deliberately narrow:

SGLang and vLLM never needed to listen on a public interface. From the Mac, the model looked like a local OpenAI-compatible API. Underneath, the requests crossed an encrypted tunnel to a loopback-only service on the workstation.

§2The model: Qwen 3.8 27B in NVFP4

The main target was a dense Qwen 3.8 27B checkpoint prepared in ModelOpt NVFP4 for the RTX 5090. The format matters. A single 32 GB GPU has to hold more than model weights: runtime workspaces, CUDA graphs, recurrent state, a speculative drafter when one is used, and the KV cache that grows with the conversation.

Making the weights fit was therefore only the first gate. The useful question was how much room remained for context and acceleration after the whole serving recipe loaded.

The launchers also carried the model-specific reasoning and tool-call parsers. A generic template can return fluent text while quietly breaking structured tools or multi-turn agent work, so every candidate had to pass text generation, recall, tool calls, vision and repeated stability checks.

§3Four recipes, not one “best” server

vLLM and SGLang solved different parts of the problem. I kept four named profiles instead of forcing every workload through one set of flags.

All four used a single active sequence and FP8 KV cache. D-Flash proposed several future tokens through a separate draft model. DSpark used a target-matched drafter with a different memory and acceptance profile. Native MTP used Qwen’s own prediction heads, avoiding a second model. The baseline deliberately removed speculation so every speed-up had a sober comparison.

§4The matched speed results

The main cross-runtime comparison used the same local benchmark suite at 32,768 context, concurrency one, FP8 KV cache and a warm loaded server. Each profile had to return a real completion before the suite started.

D-Flash 2 won the matched 32K decode comparison. DSpark was slower but used roughly 2.7 GiB less VRAM after the suite and could serve the longer contexts I wanted for agents. Native MTP sat between speculative SGLang and ordinary vLLM without needing an external drafter.

A later isolated vLLM tuning pass moved native MTP from three draft tokens to four. Under that pass’s matched conditions, decode improved from 112.8 to 120.5 tokens per second, a 6.8% gain, with unchanged outputs. I keep that result separate from the table because it came from a later A/B rather than the original cross-profile suite.

I also excluded an older 203.1 tokens-per-second D-Flash run from the headline comparison. It used a different 512-token test. It was real, but it was not matched.

§5Context length is a KV-cache decision

The KV cache stores the attention state associated with tokens already processed so the model does not recompute the whole conversation for every new token. More context means more cached state. More concurrent requests multiply the pressure. A drafter, graphs and runtime workspaces compete for the same VRAM.

That produced three context numbers that the dashboard now tracks separately:

• Requested context: what the launcher asks the runtime to reserve.
• Declared model length: what the model API reports as its maximum.
• Physical KV pool: what the loaded runtime can actually hold on this GPU.

The effective limit is the smaller of the declared length and the physical KV pool. A model endpoint can advertise a large number while the actual loaded configuration has less space.

This is why the fastest recipe was not automatically the longest. D-Flash 2 remained my validated 32K speed profile. DSpark’s lower memory footprint allowed 112K. The target-only vLLM recipe could reserve the largest context because it did not carry a separate drafter.

§6What the long-context tests actually reached

I used progressive three-needle recall tests and recorded the real prompt length, not only the configured ceiling.

The vLLM profile was configured for 262K, but the largest measured prompt contained 178,014 tokens. I do not call that a full 262K recall proof. It does show the practical cost curve: context that fits can still carry a substantial time-to-first-token penalty.

§7The optimisation that made everything worse

One of the best lessons came from an apparently sensible change. I increased DSpark’s prefill chunk to 8,192 tokens, expecting fewer chunks and faster prompt processing.

Instead, the larger reservation disabled the draft model’s CUDA graph. Decode collapsed from roughly 109 to 19 tokens per second even though speculative acceptance still looked healthy. Increasing static memory restored the graph, but it reduced the intended 114,688-token physical KV pool to 89,528 tokens.

I rejected both configurations.

The useful setting was a 2,048-token prefill chunk. In a later matched pass it reduced median 8K-prompt TTFT from 0.722 to 0.645 seconds and 35K-prompt TTFT from 4.286 to 3.837 seconds. Measured prefill rose from 8,129 to 9,081 tokens per second—an 11.7% gain—while decode stayed approximately flat at 108.7 tokens per second.

§8Tokens per second is not answer speed

Decode throughput is useful, but it can reward the wrong thing. A reasoning-heavy configuration can emit speculative reasoning tokens quickly and still take longer to produce the answer a person is waiting for.

The measurement stack therefore separates prompt prefill, first visible token, decode time, total time to a finished answer, speculative acceptance and task correctness. In chat tuning, some thinking runs reported 240–285 decode tokens per second while a roughly 115-token-per-second non-thinking configuration finished sooner.

Prefix caching produced a larger practical gain than most flag changes. Repeating an approximately 16K shared prefix reduced TTFT from 2.891 seconds cold to 0.283 seconds warm—a 90.2% reduction. Stable system prompts matter because a timestamp or session identifier inserted into the prefix can invalidate that cache.

§9A fast server that disappears is not a server

The next failures were operational. A process launched through an interactive remote session can die when that session closes. WSL was also terminating the inference runtime at idle boundaries, and its GUI layer was crashing in the background.

The final setup used allowlisted named profiles, persistent startup, automatic tunnel reconnection and a WSL configuration that kept the compute instance alive while disabling the unused GUI layer. The workstation was configured not to sleep or hibernate while plugged in.

Readiness remained stricter than process liveness. The system distinguished configured, starting, ready, degraded, failed and stopped states. A listening port was never enough: the gate required model identity and a bounded real completion.

§10From home LAN to Tailscale

The LAN was a good bootstrap path, but it could not solve access when I left home. I did not want to forward SSH or inference ports through the router.

Codex installed Tailscale on the Windows workstation through the existing LAN SSH session. Device enrolment used a visible, already authenticated browser flow; no password, one-time code, pairing secret or private key was placed in a script or repository. A security warning then prompted an update of both clients before the remote path was accepted.

The persistent SSH tunnel was retargeted from the workstation’s LAN identity to its private Tailscale identity. While both machines were still at home, Tailscale correctly used the local network as transport, but the SSH destination was already the encrypted private address.

That was not yet off-site proof.

§11The test that completed the build

I left home with the Mac and moved onto a different network. Only then did the acceptance test become meaningful.

• Tailscale showed a direct encrypted path back to the workstation.
• A completely fresh SSH connection passed strict host verification.
• The persistent inference tunnel survived the network transition.
• Windows SSH and Tailscale services remained available unattended.
• A real request returned from Qwen 3.8 on the RTX 5090.

The observed path latency was approximately 30 milliseconds. No router port was opened, and the model server still listened only on the workstation itself.

§12The operating verdict

There was no single winning runtime. There was a useful division of labour.

• SGLang with D-Flash 2 became the fast 32K chat profile.
• SGLang with DSpark became the lower-VRAM route to 112K agent and Deep work.
• vLLM with native MTP provided a simpler speculative path without a separate drafter.
• vLLM baseline remained the reliable control and rollback.
• vLLM Long traded first-token latency for deliberate 64K–262K jobs.

The workstation is now more than a powerful PC on a desk. It is a private inference node with named configurations, measured limits, a rollback path and an access model that continues working when the control machine leaves the building.

The lasting lesson is not that Codex can run remote commands. It is that remote AI-operated infrastructure needs the same discipline as any other production system: narrow authority, explicit profiles, matched measurements, real acceptance tests and proof from the network where it is supposed to work.