OpenShift AI 3.5 packages LoRA fine-tuning into a Ray workflow

Red Hat has published a worked path for fine-tuning a language model with Low-Rank Adaptation on OpenShift AI 3.5, using Ray for cluster execution and Training Hub for the algorithm interface. The useful part is not a new fine-tuning technique; it is the amount of infrastructure assembly the platform now removes from the notebook workflow.

What the walkthrough changes

In the Red Hat Developer guide, OpenShift AI 3.5 supplies a Ray CUDA runtime image with Training Hub preinstalled. That lets a team run supervised fine-tuning, offline supervised fine-tuning, LoRA and group relative policy optimization without installing the algorithm package into the training environment. Red Hat says the packaged runtime can also operate in an air-gapped environment when the model and dataset have been staged on cluster storage.

The example fine-tunes Qwen2.5-1.5B-Instruct for SQL generation. A Jupyter workbench prepares the model and data, defines a Ray cluster through the CodeFlare SDK and submits a RayJob; the training itself runs in the Ray cluster rather than in the workbench. KubeRay creates the cluster, executes the job and removes the resources after completion.

That separation is the main architectural point. The notebook acts as a control plane, shared storage carries models, datasets and checkpoints between components, and GPU capacity belongs to the short-lived training cluster.

Who should care

The guide is aimed at teams already using Ray or building multi-stage AI pipelines. Red Hat contrasts Ray's explicit cluster configuration and elastic scaling with the more automatic, static TrainJob model available through Kubeflow Trainer. Both backends consume the same Training Hub algorithms and produce interchangeable fine-tuned models, according to the walkthrough.

The storage requirement deserves attention before anyone copies the notebook. The workbench and Ray pods mount the same persistent volume simultaneously, so the example requires a storage provisioner that supports ReadWriteMany access. It also assumes OpenShift AI 3.5 or later, enabled Ray components and an NVIDIA GPU; Red Hat recommends A100 or H100 hardware for the documented setup, while its sample LoRA cluster uses one GPU.

What to try

Red Hat links complete examples for all four supported algorithm paths in its AI examples repository. For a first test, the LoRA notebook uses a public text-to-SQL dataset and checks whether fine-tuning changes the model from explanatory, Markdown-wrapped answers to bare SQL suitable for an application interface.

Teams evaluating the path should first confirm RWX storage and GPU scheduling, then clone the example into an OpenShift AI workbench. The important production questions come after the demo: how models and datasets enter an air-gapped cluster, how checkpoints are retained after the Ray job is removed, and which backend—Ray or Kubeflow Trainer—fits the team’s existing operational model.