vLLM adds day-zero serving support for Meta's 816B-parameter Muse Spark Open

vLLM has announced day-zero support for Meta's Muse Spark Open, an open-weight mixture-of-experts model aimed at long-horizon coding-agent workloads. The model has 816 billion total parameters, activates 42 billion per token and supports a one-million-token context window.

The vLLM engineering post describes deployment paths for NVIDIA and AMD accelerators, initial TPU work, and a DSpark speculative-decoding model intended to raise serving throughput. It also carries an important availability caveat: the main implementation is still under review in vLLM pull request 61879. A nightly PyPI wheel is expected only after that code merges, which the project says usually takes about a week.

What the first integration supports

Meta released BF16 and NVFP4 weights, while AMD released MXFP4 weights. vLLM provides prebuilt serving images for NVIDIA and AMD. Tensor and pipeline parallelism are available on both vendors; NVIDIA additionally gets expert parallelism, tensor-and-expert parallelism, decode expert parallelism and prefill/decode disaggregation. ROCm support currently covers tensor and pipeline parallelism on AMD Instinct MI355X, including FP8 and MXFP4 formats.

The model's architecture helps explain the deployment demands. All 62 decoder layers are MoE layers with 256 experts each. Layers alternate full attention with sliding-window attention in a four-layer cycle: three layers look at a 2,048-token window and the fourth sees the full context. vLLM says that structure cuts the one-million-token KV cache to slightly more than one quarter of an all-full-attention design.

Muse Spark Open also accepts images, video and audio. Images and video frames use the same roughly two-billion-parameter Perception Encoder as Muse Glimmer; audio runs through a separate 118-million-parameter encoder. Video audio is not processed automatically, so applications that need both must send the video and an extracted audio stream as separate inputs.

DSpark and the TPU boundary

The integration includes a DSpark draft model for CUDA and ROCm. DSpark uses block diffusion to propose as many as seven tokens in one pass, with confidence-based scheduling and adaptive verification. Operators must enable it explicitly in vLLM's speculative configuration.

Initial TPU support sits behind a narrower boundary. vLLM says it is built on TorchTPU and directs users to contact the project or Google for code access. The post reports more than 650 tokens per second on four TPU v7 chips and more than 960 on eight with DSpark at an acceptance length of six. Those figures are project-reported results, not a cross-platform comparison under the same deployment recipe.

What operators should do

Teams testing immediately can start with the vendor container images and the documented serve command, but should pin those images and treat the integration as pre-merge software. PyPI-based workflows need to wait for pull request 61879 to land and for the nightly wheel to include it.

The practical signal is breadth rather than maturity: vLLM has a path for an unusually large multimodal MoE model across the two major GPU stacks, a speculative decoder and early TPU work. Production readiness will depend on the pending merge, hardware-specific parallelism limits and workload testing at the context lengths an application will actually use.