A lawyer who built a private AI server around Nvidia V100 GPUs has expanded it to 12 cards, allowing every expert in DeepSeek V4.1 Flash to run at 4-bit NVFP4 precision. Nick Mermiges documented the work on his GitHub repo and in a longer write-up on his site, describing the third board of V100s added to his existing setup. The machine now runs all 15,360 experts without further compression, a goal he says he intended from the start.

The expansion addresses a memory shortfall. The original eight 32GB cards offered 254 GiB, while the DeepSeek V4.1 Flash experts need 278.9 GiB under Nvidia’s NVFP4 format. His previous workaround kept every expert on the cards at 2-bit precision and converted the most-used 5,949 experts, about 38.7%, to NVFP4 until memory filled up. With the third board in place, he reports the expert values match those in DeepSeek’s own release, though other parts of the model use compact forms.

The V100 lacks FP4 and FP8 tensor cores, so each NVFP4 number must be unpacked in software into half precision. Mermiges notes the compact experts still pay off because the cards read fewer bytes, not because they compute faster. That explains how a 2018-era card can handle a format from the Blackwell era.

The model’s layers are split across the boards as 0 - 13, 14 - 27, and 28 - 39, with each board’s section divided among its four cards. Cards on a board communicate over NVLink, while hand-offs between boards cross the host’s PCIe through two switches. The first two boards share a PEX88096 switch, keeping their hand-off inside it, but the third board sits behind a separate PLX 8749, so the second hand-off crosses the host’s PCIe bridges and adds latency.

Power and clock settings were also changed, with each card’s power cap raised from 150 W to 175 W and its clock cap lowered from 1,522 MHz to 1,452 MHz. Mermiges measured the first code token at 256K context at 58 seconds, roughly twice as fast as the 108.4 seconds after his previous upgrade. Prompt reading reaches about 4,500 tokens a second past 32K, more than double the roughly 2,200 tokens a second of his first build.

Code decode is 23% faster at 1K but shows no real gains at 32K and above, so the main benefit is a shorter wait for the first token. Running three requests at 128K produced about 360 tokens per second in total on code and 213 on legal; Mermiges cautions that the setup is not a batched-decode engine and that more requests than sections gain little. He said a day-long attempt to push each card to 200 W caused the host to trip once under four long chats at once.

Four optimizations were disabled after missing their targets: a conversation store that saves to disk, a leaner hand-off between sections, tighter expert packing, and exact four-card-sum arithmetic. Mermiges said he plans to study batched decoding and to enable the conversation store so a matter’s documents survive a restart. He acknowledged the numbers do not establish answer quality in use, but said the setup has been his default DeepSeek machine since Monday and that the model is the first one that works like Opus 4.6 for him.

More hardware news from TechManNews.