Red Hat’s Redis quickstart puts routing ahead of expensive inference

Red Hat has published a runnable OpenShift AI quickstart that attacks inference cost before a request reaches an expensive model. The pattern combines a Redis semantic router with a semantic cache, then demonstrates the design through an insurance-assistant application, notebooks and deployment manifests.[^1][^2]

What the request path does

The router classifies incoming text into three paths: blocked, simple or complex. Off-topic requests leave without consuming model tokens; simple requests go to a cheaper non-reasoning model; complex requests proceed toward the full agent. For the expensive path, the service checks Redis for a semantically similar approved response before generating a new one.[^2]

That ordering is the important design choice. It separates three decisions that are often collapsed into one LLM call: whether the request belongs in the application, how much model capability it requires and whether the answer already exists. Red Hat’s accompanying blog describes Redis as both the classification layer and the semantic memory, while OpenShift AI serves embedding models and the LLM integration.[^1]

What the quickstart includes

The documentation provides a Helm-based deployment, a five-tab dashboard and notebooks for a baseline agent, router-plus-cache flow and asynchronous production queue. The production view distributes work across horizontally scalable workers backed by Redis for queues, memory and cached data. A GPU is not required for the quickstart.[^2]

The example also makes cache admission explicit: after a complex request reaches the agent, positive user feedback can save the response with a time to live. That is safer than treating every generated answer as automatically reusable, but it is still only a starting policy. Teams will need stronger approval, invalidation and provenance controls where answers depend on changing policy or customer-specific state.

What to measure

Red Hat links the architecture to lower token spend and faster responses, but the implementation should be evaluated against each workload. Measure the percentage of traffic blocked, sent to the small model, served from cache and escalated to the full agent. Track false route decisions and stale cache responses alongside latency and token cost.

The blog cites a partner customer example for an “up to 4X” speed improvement rather than a benchmark of this quickstart.[^1] Practitioners should not treat that figure as a guaranteed result. The more durable takeaway is the testable architecture: classify first, reuse only approved answers and reserve costly reasoning for requests that need it.

[^1]: Red Hat, “Stop paying for the same prompt: Optimize AI costs with Redis on Red Hat OpenShift,” Aug. 18, 2026. [^2]: Red Hat Documentation, “Optimize insurance agent LLM costs,” accessed Aug. 18, 2026.