llm-d 0.9.0 separates batch, Intel XPU and autoscaling decisions

llm-d 0.9.0 does not introduce one universal inference configuration. It packages several distinct deployment paths: a new batch-gateway image and chart, a new SGLang image for Intel XPU, and KEDA-oriented autoscaling patterns. Platform teams should treat those as separate architecture choices rather than enabling all three during a routine version bump. The release was published Aug. 17.

What changed

The component manifest adds llm-d-batch-gateway 0.5.0 as a new image and Helm chart, and adds llm-d-xpu-sglang 0.9.0 as a new image. The wider stack also moves its packaged SGLang server from 0.5.13.post1 to 0.5.16, while the workload-variant autoscaler advances from 0.8.0 to 0.9.0. The same release updates router endpoint-picker and disaggregation-sidecar components from 0.9.0 to 0.10.0, so an operator should validate the release as a coordinated component set, not infer compatibility from the top-level 0.9.0 tag alone. The component table lists each version independently.

The batch gateway changes the workload boundary. It exposes OpenAI-compatible /v1/batches and /v1/files APIs for offline jobs such as evaluations, embeddings and dataset processing, while allowing that work to use spare accelerator capacity alongside interactive traffic. It is not a stateless replacement for the existing router: the documented path requires a working llm-d router and inference pool, PostgreSQL 12 or later, Redis 6 or Valkey 8 or later, and either S3-compatible storage or a ReadWriteMany volume. The batch guide also describes tenant-separated jobs, files and results.

Who should choose which path

Teams with offline jobs and the required stateful services can add the batch gateway as a separate submission and job-management layer. Interactive-only deployments gain little from taking on its database, cache and shared-storage dependencies.

The new XPU-SGLang image gives Intel accelerator users a packaged SGLang path, but it does not erase cluster prerequisites. llm-d’s accelerator documentation limits the described Intel path to Data Center GPU Max 1550 and BMG GPUs and requires Intel Resource Drivers for Kubernetes. RDMA-based prefill/decode disaggregation adds an RDMA DRA driver, GPU-to-NIC PCIe alignment and UCX transport configuration. Those boundaries remain hardware and topology decisions, not image-selection details.

For autoscaling, the current development documentation distinguishes two KEDA-based designs. KEDA can scale a homogeneous pool directly from endpoint-picker queue depth and running-request metrics. The workload-variant autoscaler instead calculates desired replicas across heterogeneous model variants, using signals such as KV-cache utilization, queue depth and performance budgets; KEDA or an HPA then acts on that signal. The documentation marks the older VariantAutoscaling custom resource as deprecated since llm-d 0.8.0.

Upgrade boundaries

Before moving, inventory the exact component and model-server image versions, test router and sidecar changes together, and choose one autoscaling control loop per pool. Batch adopters should provision and back up the gateway’s database and object or shared-file storage before directing production jobs to it. XPU adopters should verify the documented device-driver and, where applicable, RDMA prerequisites on the target cluster. The autoscaling and accelerator pages are explicitly labeled development documentation, so operators should pin the 0.9.0 artifacts they validate and recheck the versioned guidance before rollout.