Red Hat turns enterprise AI hosting choices into an operations checklist

Red Hat has published a practical architecture guide for teams deciding where enterprise AI workloads should run and what they must operate after deployment. The useful part is not a new product claim so much as the operating model: managed APIs, self-hosted models and hybrid designs move different costs, data boundaries and failure modes onto different owners.

Start with the boundary, not the model

The guide separates managed AI services from models hosted on infrastructure an organization controls. Managed APIs reduce the need to operate GPUs, model storage and serving infrastructure, but leave availability, pricing, throttling and model updates with the provider. Self-hosting keeps prompts and data within the organization’s environment and can make high-volume costs more predictable, while adding GPU capacity planning, patching and incident response.

Red Hat’s recommended hybrid pattern is explicit rather than accidental: teams should decide which workloads can call an external endpoint, which require internal inference, where embeddings and retrieval stores live, and how traffic crosses those boundaries. That framing gives architects a concrete review question before an application reaches production.

What OpenShift AI is expected to operate

For self-hosted deployments, the article positions OpenShift as the Kubernetes substrate for GPU scheduling and team isolation, with OpenShift AI handling model lifecycle, serving and agent integration. It describes ModelCar images or object storage for model artifacts, a model registry for versions and promotion state, and built-in serving runtimes based on vLLM, with llm-d for demanding distributed inference.

The integration layer is split in two. An AI gateway controls access, quotas and token budgets on shared model endpoints. For agents calling enterprise systems, Red Hat describes curated Model Context Protocol servers and an MCP lifecycle operator. A Connectivity Link MCP gateway is also described for authentication, routing and tool-level access control, but the article labels that gateway a technology preview rather than a production-ready promise.

Promote the whole AI system

The strongest operational advice is to stop treating the model as the only deployable unit. Red Hat recommends pinning and promoting model identifiers, prompt templates, retrieval-index versions and agent tool configurations together. That makes rollback a coordinated change instead of a search for which component drifted.

Production checks also extend beyond inference latency. RAG systems need retrieval quality and latency measurements; agents need correlated tool-call logs, allowlists, timeouts and safe failure behavior. Capacity plans should use peak concurrency and token throughput, while continuity plans should cover provider throttling, GPU loss, bad rollouts and broken tools.

For platform teams, the immediate action is to document the hosting choice, data path and operator for every layer before approving production use. Then test degraded modes—such as a smaller fallback model, cached responses or read-only agents—before an outage decides the behavior for you.