vLLM maps Step-5 Preview’s million-token ceiling to deployable GPU layouts

The new verified vLLM recipe for StepFun’s Step-5 Preview gives operators three materially different ways to place the model on one node, while making clear that the advertised 1,048,576-token context is a ceiling rather than a concurrency promise.

Three layouts, not three spellings

For the FP8 checkpoint on four NVIDIA B300 GPUs, the recipe offers TP4, TEP4 and DEP4. TP4 shards the model with tensor parallelism across all four GPUs and leaves expert parallelism off. TEP4 also uses tensor parallelism across four GPUs but enables four-way expert parallelism and the MegaMoE backend. DEP4 instead sets tensor parallelism to one, creates four data-parallel ranks and distributes routed experts across those ranks. The recipe says shared experts remain separate from routed experts in the expert-parallel configurations.

That distinction affects both scaling and request accounting. TP4 and TEP4 each have one data-parallel rank; DEP4 has four. The --max-num-seqs setting applies per data-parallel rank, so copying the same number between layouts does not describe the same aggregate concurrency.

The same choices extend to Hopper, but the hardware count changes. The recipe calls for eight H200 GPUs for the FP8 checkpoint and says to omit the Blackwell-specific MegaMoE flag. Four B300s and eight H200s are documented as distinct launch configurations rather than interchangeable layouts.

Quantization buys weight headroom, not free context

The NVFP4 checkpoint is designed for four B300 GPUs and is not wholly four-bit: the recipe says the package contains a mix of NVFP4 and BF16 layers, with quantization detected from the checkpoint configuration. TP4 can use automatic MoE backend selection, while TEP4 and DEP4 use MegaMoE for the routed experts.

The lower weight footprint leaves more device memory available, but the supplied NVFP4 example still caps --max-model-len at 320,000 tokens rather than one million. That is the operational caveat in the recipe: model weights fitting in GPU memory does not mean every request can consume the model’s architectural context limit.

The KV cache is the real limit

Step-5 Preview’s one-million-token figure covers input and output together. vLLM explicitly tells operators to lower context length or concurrency when KV-cache memory is insufficient. The practical reading is straightforward: the recipe verifies supported launch paths and parallel topologies; it does not establish one-million-token service at the advertised batch size. Capacity planning still requires measuring the remaining KV cache after loading the chosen checkpoint and then balancing maximum sequence length against the number of simultaneous requests.