PyTorch Conference puts vLLM’s serving architecture and hardware portability on the agenda

PyTorch Foundation has published the vLLM program for PyTorch Conference North America 2026, outlining two days of technical sessions that put production inference architecture, KV-cache movement and accelerator portability at the center of the project’s October agenda.

The program matters beyond the conference calendar because several sessions describe work already integrated upstream or concrete changes to vLLM’s serving abstractions. Red Hat engineers feature prominently in talks on attention, disaggregated serving and multimodal caching.

Red Hat engineers take on attention and KV transfer

Red Hat’s Lucas Wilkinson and Matthew Bonanni will present an overhaul of vLLM’s attention abstractions. The session is set to cover attention backends, KV-cache connectors and the hybrid memory allocator, with an emphasis on making sliding-window, sparse, compressed and hybrid attention architectures easier to support.

A separate session from Mistral AI, Amazon and Red Hat engineer Zhanqiu Hu will examine KV-cache transfer between prefill and decode workers. Its agenda includes heterogeneous tensor parallelism, bidirectional transfer, the KV Push connector and cache leases intended to improve reliability in disaggregated deployments.

Red Hat engineers Ricardo Noriega and Alex Brooks will also present Automatic Prefix Caching for stage outputs in vLLM-Omni. The proposed design aligns CPU-side tensor caches with vLLM’s block management and discovers cacheable tensors dynamically for multi-stage models.

Tiered offload moves upstream

The program describes IBM’s native tiered KV-cache offloading framework as newly integrated into vLLM without external dependencies. The design routes transfers through CPU memory as a common transport hub, consolidating I/O through a CPU buffer while remaining independent of cache layout, attention backend and accelerator topology.

Other sessions extend the portability theme. IBM and Meta will discuss hardware-agnostic model definitions intended to run across Intel Gaudi and IBM Spyre without maintaining hardware-specific model forks. Meta and Google speakers plan to show PyTorch-native TPU backends for both vLLM and SGLang, while an NVIDIA session will cover adding and removing expert-parallel workers under live traffic.

What to watch

The conference program is a preview rather than a release announcement, and several performance figures are speaker-reported claims that still need their full talks and artifacts for evaluation. Even so, the lineup shows where vLLM contributors are concentrating: separating model definitions from hardware paths, moving increasingly large caches between serving stages, and keeping inference available while worker topology changes.

PyTorch Conference North America is scheduled for October 20–21 in San Jose. The published program includes technical talks, demonstrations, lightning talks and a vLLM birds-of-a-feather session.