Red Hat breaks a production RAG workflow into five retryable OpenShift AI components

Red Hat has published a production-oriented reference workflow for retrieval-augmented generation on OpenShift AI, replacing a single Ray Data script with five independently runnable pipeline components. The design is aimed at teams that need repeatable runs, isolated retries and a record of the parameters used to build a vector collection.

The important shift is not a new retrieval algorithm. It is the move from a monolithic document-processing job to an orchestrated system that can fail, resume and evolve without rerunning every stage.

What the pipeline changes

The reference architecture uses OpenShift AI Pipelines to coordinate two parallel chains. The data chain parses and chunks PDFs with Docling on Ray, optionally deploys an embedding service, and writes vectors to Milvus. The model chain downloads an LLM and deploys it through KServe and vLLM.

Those chains are split into five Kubeflow Pipeline components: parse_and_chunk, ingest_to_milvus, optional deploy_embedding_model, download_model, and model_deployment. Each has its own image, parameters and interface, so teams can replace a parser or reuse the model-deployment stage without editing the whole workflow.

Between parsing and ingestion, the design writes JSONL output to S3-compatible storage. That is a deliberate property of this implementation rather than an automatic OpenShift AI feature. If vector ingestion fails after a large document set has already been parsed, the operator can repair the Milvus connection and rerun only the failed component instead of repeating the expensive parse.

Where OpenShift AI fits

The example combines Docling, Ray through KubeRay, Milvus, S3-compatible storage, vLLM and KServe. OpenShift AI supplies the managed orchestration layer around them: pipeline run history, Ray cluster lifecycle, GPU scheduling through Kueue, model serving, and the dashboard used to launch and inspect runs.

The components also expose practical scale choices. Smaller collections can use the Granite 125 million-parameter English embedding model on CPUs, while larger jobs can direct embedding requests to a GPU-backed vLLM service. Model downloads are cached on persistent storage with a sentinel file so subsequent runs do not fetch the same weights again.

What to try

Platform teams evaluating the pattern need an OpenShift AI cluster with AI Pipelines, KubeRay, Milvus, object storage, access to the selected Hugging Face model and GPU capacity for serving. Red Hat’s example repository includes a notebook that configures credentials and parameters, compiles the pipeline and submits it.

The practical test is failure recovery: run the pipeline on a representative document set, interrupt or deliberately break the Milvus stage, then confirm that ingestion can resume from the stored JSONL output without repeating Docling processing. That exercise will show whether the component boundaries and storage choices match the team’s operating model before the workflow is adapted for production data.