Red Hat maps the four layers needed to move enterprise AI beyond a model pilot

A model endpoint is only one part of an enterprise AI system. In a new architecture guide, Red Hat divides the production stack into four layers: compute, model storage, inference and serving, and integration with existing systems.

The framework is useful because it makes the boundary between managed and self-hosted AI explicit. A managed model provider can operate the first three layers, but the application team still owns integration. A self-hosted deployment keeps all four layers inside the organization’s operating model.

What the four layers cover

The first layer is accelerator-backed compute. Red Hat describes GPU capacity, device memory and expected concurrency as linked sizing decisions: insufficient capacity appears as latency, throttling or queued requests. OpenShift and GPU operators are presented as the scheduling and isolation layer for self-hosted inference.

The second layer is durable model storage. That includes weights, tokenizers and serving configuration, with object storage or versioned OCI images as common packaging choices. The guide recommends maintaining an approved catalog of model checkpoints rather than treating shared storage as an informal model registry.

The third layer separates the inference engine from model serving. Engines such as vLLM load weights, tokenize requests, batch work and stream output. Serving adds the operational envelope: routing, health checks, replicas and autoscaling aligned to GPU capacity. Red Hat positions Red Hat AI Inference, built around vLLM, as its supported runtime option and says equivalent capabilities are included with Red Hat AI Enterprise.

Integration is the lasting responsibility

The fourth layer is where the model connects to databases, identity providers, monitoring systems, internal APIs and business workflows. Red Hat distinguishes two traffic directions.

For systems calling AI, the application remains in control and sends requests to an OpenAI-compatible endpoint. Production concerns include authentication, retries, quotas, token-aware rate limits and audit logging. The guide points to Red Hat Connectivity Link as an AI-aware gateway for this traffic on Kubernetes and OpenShift.

For AI calling systems, an agent or its framework selects tools and may chain actions. That expands the control problem to tool discovery, delegated access, input validation and unintended actions. The guide presents Model Context Protocol as a common tool interface, while warning that enterprises still need federated discovery, identity-based filtering and audit. Red Hat says Connectivity Link’s MCP gateway, currently in technology preview, is intended to provide that governed front door.

What teams can do with the framework

Architecture reviews can use the four layers as a responsibility checklist. Teams choosing managed APIs should document which provider operates compute, storage and serving while retaining an explicit plan for integration. Teams self-hosting on OpenShift should add model artifact governance, accelerator capacity and serving operations to that same plan.

The practical lesson is deliberately unglamorous: selecting a model does not select an architecture. The work that turns a capable model into a dependable application sits in the infrastructure and integration around it.