llm-d 1.0 stabilizes the control plane for shared inference fleets

llm-d has released version 1.0.0, turning several production-operability projects into the first major release of the distributed inference stack. The release aligns most core components on a common 1.0 version and makes the router configuration API stable at v1.

What changed

The largest change is multi-tenant flow control. llm-d 1.0 adds priority- and fairness-aware admission with controlled load shedding, including probabilistic admission, program-aware fairness and turn-priority scoring. The intent is to let latency-sensitive requests and bursty agent traffic share a fleet without one class starving another under overload; those settings now sit under a consolidated flowControl configuration.

Operations also get a firmer high-availability path. The router can run with hot standby, while DisaggregatedSet coordinates prefill and decode replicas as one logical unit. Its rollout model includes lockstep upgrades, readiness based on pod snapshots and native revision routing, all aimed at avoiding service interruption during changes.

For agent workloads, the release adds a session-state-producer plugin and evolving session affinity so multi-turn requests can stay close to cached context. The Responses API now supports previous_response_id, and turn-priority scheduling gives operators another control for conversations that arrive as linked requests rather than isolated prompts.

The project also added two ecosystem components: llm-d-api-extensions, which exposes inference-server configuration, hardware and KV-cache metadata to routing and scheduling extensions; and llm-d-pd-utils, which provides preflight diagnostics, GPU topology discovery and network benchmarks for distributed inference deployments.

Who it affects

Platform teams already operating prefill/decode disaggregation have the most immediate work. The release notes say those guides now require LeaderWorkerSet 0.11.1 or newer with the DisaggregatedSet API enabled, and recommend moving to LeaderWorkerSet 1.0.0. Older installations below that floor will not match the 1.0 guide assumptions.

Image consumers also need to review their supply chain. llm-d has deprecated its own llm-d-rocm image builds and says accelerator partners are moving toward their upstream inference-engine images. The 1.0 release temporarily mirrors a vLLM 0.32.0 release-candidate ROCm image to guard against image garbage collection, while the component matrix otherwise advances several vLLM, SGLang and TensorRT-LLM bases.

The version-alignment promise has limits. Most core components move to 1.0.0, but the release matrix deliberately lists exceptions, including the XPU SGLang image and third-party engines. Operators should treat the published matrix, not the umbrella tag alone, as the compatibility record.

What to do

Before upgrading, compare the deployed component and model-server image versions with the two matrices in the release notes. Confirm LeaderWorkerSet meets the new minimum, enable DisaggregatedSet where the deployment depends on the updated guides, and test router failover plus rolling updates against the project's linked release-readiness matrix.

Teams using the ROCm image should plan away from the deprecated llm-d-built stream and verify the replacement upstream image for their disaggregation path. Multi-tenant environments should stage the new flowControl policy under representative overload rather than copying defaults directly into production.

The release notes are unusually candid that not every 1.0 goal was completed. That makes the stable API and aligned component versions useful boundaries, but not a declaration that every incubating part of the ecosystem is equally mature.