vLLM’s merged Hy4 path combines sparse attention, MTP and tool-aware serving

vLLM has merged a 36-commit serving path for Tencent’s Hy4 preview model, bringing the model’s sparse-attention architecture, native multi-token prediction and tool-aware output handling into one deployment recipe. The important qualification for operators is timing: the code landed on main on August 29, after vLLM 0.28.0 was published on August 26, so the merge is not evidence that an ordinary 0.28.0 installation supports the model.

What changed

The merged pull request adds Hy4 as a supported model and documents a serving command that enables the FLASHMLA_SPARSE attention backend, MTP speculative decoding, and Hy4-specific reasoning and tool-call parsers. It spans 47 changed files and more than 6,000 added lines, reflecting work across model integration, kernels, parsing and deployment documentation rather than a small registry update.

Those pieces correspond to the model’s architecture. Tencent’s Hy4 model card describes a 770-billion-parameter mixture-of-experts backbone with 49 billion parameters activated per token, Gated DeepSeek Sparse Attention with cross-layer index reuse, and a native MTP layer. The same model card says Hy4 has a one-million-token context window and publishes both full and FP8 weights under Apache 2.0.

For serving operators, the merged path matters because it connects three otherwise separate concerns. Sparse attention needs an implementation that follows Hy4’s index-selection design; MTP needs the speculative-decoding path to consume the model’s native prediction layer; and agent workloads need model-specific parsing so reasoning text and tool calls reach OpenAI-compatible clients in the expected fields. The documented command enables all three together and exposes Hy4 through vLLM’s OpenAI-compatible server.

Availability is narrower than the merge

The current general vLLM release remains v0.28.0, published three days before the Hy4 pull request merged. Operators should therefore not read “merged” as “included in v0.28.0.”

Tencent instead points users to the model-specific vllm/vllm-openai:hy4-preview container image, while the pull request also documents building vLLM from the repository source. The model card’s example uses eight-way tensor parallelism with the FP8 checkpoint, three MTP speculative tokens, FLASHMLA_SPARSE, and the hy_v4 reasoning and tool-call parsers.

What operators should do

Teams evaluating Hy4 should pin the model-specific image or a tested source commit rather than assume support from the latest numbered vLLM release. They should also retain the full documented flag set during validation: dropping the sparse backend, MTP configuration or parsers changes the serving path being tested. Production adoption should wait for a normal release that explicitly contains the merge, or treat the special image as a separately versioned artifact with its own qualification and rollback process.