AgentX 1.0 brings long-context agent workloads into open inference benchmarking

SemiAnalysis has released AgentX 1.0, an Apache 2.0 benchmark intended to measure the long-context, multi-turn inference patterns produced by coding agents rather than the fixed prompt-and-response workloads common in earlier accelerator tests. The launch article says the benchmark replays anonymized coding-agent traces with contexts reaching one million tokens and includes bursts from sub-agents, tool calls and high prefix reuse.

The work is relevant to the Red Hat AI ecosystem because it exercises vLLM and the distributed-inference components around it. SemiAnalysis credits contributors from both vLLM and Red Hat’s llm-d project, and says the benchmark has already guided upstream optimization work across vLLM, SGLang, TensorRT-LLM, LMCache, Mooncake and related projects.

What the benchmark changes

Conventional inference tests often prescribe a fixed input and output length. AgentX instead reconstructs multi-turn request schedules from 393 anonymized Claude Code traces, according to the release description. That changes what is under stress: routing requests back to workers holding the right prefix, retaining useful KV-cache entries, moving cache data between GPU and host memory, and handling fresh contexts created by sub-agents.

Those are system-level concerns, not only measurements of accelerator kernels. The launch describes benchmark runs across more than 1,000 chips, including AMD MI300- and MI355-class hardware and NVIDIA H200, B200, B300, GB200 and GB300 systems. Results, configurations, logs and a public dashboard are linked from the project’s InferenceX repository, which identifies AgentX as its August 2026 release and lists the supported hardware.

Why platform teams should care

For teams deploying agents, the practical lesson is that throughput from a short, single-turn test may not predict the behavior of a production agent service. AgentX is designed to expose the effects of long-lived sessions, high cache reuse and bursty sub-agent traffic on latency and capacity. Its published results separately examine time to first token, per-user generation speed and end-to-end completion time rather than collapsing them into one score.

The benchmark also gives upstream projects a shared workload for tuning. SemiAnalysis says its collaborators used the trace replayer to drive fixes in hybrid-attention prefix caching, KV-cache handling, routing and other paths used in production serving. The open repository contains the runner, configurations and continuous benchmark infrastructure, while the underlying article links the trace dataset and individual upstream changes.

AgentX is not a product certification, and its vendor comparisons will continue to move as software changes. Its more durable contribution may be the workload definition: an open attempt to test the distributed serving problems that appear when agent conversations become long, stateful and parallel.