Red Hat’s ticket-agent quickstart treats evaluation and observability as architecture, not add-ons

Red Hat has published an OpenShift AI quickstart for ticket-handling agents that is more useful as an architecture reference than as another chatbot demo. The design puts ticket state, routing, tool use, human escalation, evaluation and distributed tracing into one deployable system, showing the scaffolding required around a model before an agent can touch an enterprise workflow.

A routed system, not one oversized agent

The quickstart connects the open source Zammad ticket platform to a request manager that normalizes incoming events and restores conversation state. A router agent then sends each request to a specialist. The supplied example separates laptop-refresh requests from general support questions, but the pattern is intended to accept additional specialists without turning one prompt into a monolith.

Each specialist is implemented as a LangGraph state machine. Agents can query retrieval-augmented knowledge bases, ask users for missing information and invoke Model Context Protocol tools to look up device details, close or label tickets, escalate cases, or request a manager’s approval. Persistent session memory keeps a multi-turn conversation attached to the ticket rather than treating every comment as a new request.

That separation matters operationally. Ticket labels record whether work was closed by an agent, escalated for human review or routed for approval, giving operators a basic view of automation outcomes inside the system that already owns the workflow.

Evaluation is part of the deployment

Red Hat’s reference design includes multi-turn testing with DeepEval because fixed assertions are a poor fit for non-deterministic conversations. The evaluation layer combines predefined flows with generated scenarios and scores behavior against business rules, such as whether an agent closes a ticket or escalates it at the correct point.

The same emphasis appears in observability. OpenTelemetry follows a request across the router, specialist agents, knowledge retrieval and MCP services. OpenShift AI’s MLflow integration groups traces by session so an operator can reconstruct a conversation instead of inspecting disconnected model calls.

The quickstart also includes NeMo Guardrails with TrustyAI. Red Hat presents guardrails, tracing and evaluation as working parts of the system rather than a list of production concerns left for adopters to implement later.

What teams can try

The documented deployment uses OpenShift 4.17, OpenShift AI 3.4 and an external Llama 4 Scout 17B model, with Llama 3 70B optional for evaluations. Red Hat says the hands-on path takes 60 to 90 minutes and offers a simplified testing mode before moving to Knative Eventing and Kafka for production-style scale.

The laptop workflow is only the sample domain. The reusable lesson is the control plane around it: normalize events, route narrowly, expose actions through MCP, retain session state, make human handoff explicit, and test the agent’s decisions continuously. That is the part platform teams can carry into access requests, software licensing, compliance reviews or other structured work.