vLLM Semantic Router turns multimodal inputs into typed decisions

vLLM Semantic Router has released Decision 3.0, a family of six open models ranging from 0.6 billion to 27 billion parameters. The collection is built for a narrower job than a general-purpose chatbot: it takes text or structured state, optionally adds images or video, and returns probabilities for typed decisions rather than generating free-form text.

What changed

The Decision 3.0 collection includes d3-edge, d3-lite, d3-nano, d3-mini, d3-flash and d3. Hugging Face lists the family from 0.6B through 26B-class models; the flagship d3 model card describes 26.09 billion parameters, including a 460 million-parameter vision encoder. All six releases use the Apache-2.0 license.

The interface supports three explicit question types: choosing among options, answering yes or no, and assigning a score. A request can bundle several questions over the same input, with each answer returned as a probability distribution. That makes the models candidates for routing, moderation, triage and policy gates where an application needs a machine-readable decision rather than another block of prose.

The flagship can accept text or JSON plus multiple images and videos. Its model card says video is sampled at two frames per second, with up to 32 frames spread over a clip. The project also publishes a Transformers quickstart and examples for customer-service routing, receipt inspection and video classification.

Who should care

Teams already using vLLM Semantic Router now have a model family designed around the router’s core control-plane problem: deciding where a request goes and what policy applies. The smaller variants make that pattern available where a 27B model would be impractical, while the flagship extends the decision surface beyond text.

The project reports a 65.45 score on the public Jev Decision Index for d3, 8.5 points above its Decision 2.0 result. It also reports 71.6 on the benchmark’s vision index and 77.4 percent on an internal Perception Test video evaluation. Those numbers come from the project’s own model card; independent reproduction will matter before teams treat them as deployment guarantees.

What to do

Platform teams should start with a contained routing or classification task and compare the probabilities against an existing rules engine or labeled test set. The model card requires trust_remote_code=True in its sample loader, so adopters should review the repository code and pin model revisions before production use.

The most useful test is not whether Decision 3.0 can sound persuasive. It does not try to. The test is whether its typed outputs are calibrated, repeatable and cheap enough to sit in the request path of an agent or inference platform.