OpenShift AI 3.5 EA2 puts policy and evaluation closer to agent tool calls

Red Hat OpenShift AI 3.5 EA2 moves several controls closer to the boundaries where enterprise agents call tools, run evaluations and consume inference capacity. The release is explicitly Early Access: Red Hat says it is unsupported, incomplete and unsuitable for production or business-critical workloads. That makes it a test surface, not an upgrade target.

The 3.5 release notes nevertheless show where the platform is heading. The most consequential previews connect NeMo Guardrails to the MCP Gateway, expose model evaluation through an MCP server and add asynchronous batch requests to distributed llm-d deployments.

What changed

A Technology Preview integration can enforce NeMo Guardrails at the MCP Gateway rather than requiring every agent application to implement its own checks. Red Hat says the gateway-layer policy can inspect tool-call traffic for personally identifiable information leakage, prompt injection and content-safety violations. The TrustyAI Operator discovers the target gateway and related plugin after administrators reference an MCPGatewayExtension from a NemoGuardrails custom resource.

EvalHub also gains a Technology Preview MCP server. Compatible coding agents can discover evaluation providers, submit jobs, monitor or cancel runs and use guided prompts for comparing evaluations. The dashboard separately adds pass/fail thresholds for individual benchmarks and suites, while Kueue integration can apply quotas, priority and admission control to evaluation jobs.

For inference, llm-d adds an OpenAI-compatible /v1/batches API as a Technology Preview. Batch requests run asynchronously and at a lower priority during periods of low cluster activity, with the stated goal of preserving service levels for interactive traffic. The model catalog can also display validated vLLM arguments for models whose tool-calling configurations Red Hat has tested.

Other EA2 changes include GPU-accelerated Docling images for batch and on-demand document conversion, prompt versioning through the MLflow prompt registry, OGX multi-tenancy and operational model-catalog data such as cold-start load time and minimum vRAM. The Responses API is labeled generally available on OGX, although the surrounding 3.5 EA2 distribution remains an unsupported Early Access release.

Who should test it

The release is most relevant to platform teams already evaluating governed agent access, shared inference or model-evaluation services on OpenShift AI. It provides a way to test whether gateway enforcement can replace duplicated guardrail logic and whether evaluation jobs can fit existing cluster scheduling and quota policy.

Teams operating llm-d can also test whether low-priority batches improve accelerator utilization without disturbing latency-sensitive endpoints. Red Hat does not publish performance results in the notes, so operators will need their own measurements for queue delay, interactive latency and GPU occupancy.

What to do

Do not treat 3.5 EA2 as production software. Use a separate non-critical cluster, record the custom-resource schemas you exercise and expect them to change or disappear.

For MCP testing, begin with a narrow tool set and verify both allowed and blocked calls at the gateway. For EvalHub, test cancellation, threshold behavior and Kueue admission under contention. For batch inference, measure the effect on live traffic instead of assuming lower scheduling priority is sufficient isolation. Those checks will reveal whether the previews solve an operational problem before the interfaces harden.