Red Hat makes OpenShift AI notebook image builds reproducible and offline

Red Hat engineers have documented how Open Data Hub and Red Hat OpenShift AI moved notebook and runtime image builds to a hermetic model: every dependency is resolved and cached before the container build begins, then the build runs without network access.

The result is more than an air-gap workaround. It is a reusable supply-chain pattern for teams that need the same image recipe to work locally, in GitHub Actions and in Konflux while producing dependency records that release policy can inspect.

What changed

The pipeline separates dependency acquisition from image construction. Teams commit lockfiles, prefetch RPMs, Python wheels, npm packages and Go modules into a local cache, and then let Podman or Buildah install only from that cache. The same Containerfile can consume a cache prepared by a local script, a GitHub Actions workflow or Konflux's Tekton prefetch task.

The Open Data Hub guide says dependencies are pinned by URL and SHA-256 checksum. That makes builds less sensitive to mirror outages and upstream version drift, while giving software bills of materials recognizable package ecosystems instead of a collection of opaque downloads.

Upstream and product builds deliberately retain separate lock trees. Open Data Hub uses its upstream dependency set, while subscribed Red Hat bases use downstream lockfiles so CentOS packages are not mixed into RHEL-based images.

Why packaging shape matters

The engineering lesson is that an offline build is not automatically a policy-compliant build. Red Hat's article says generic release tarballs can satisfy network isolation but still fail packaging expectations or leave poorer SBOM data.

The preferred path is to express dependencies through typed ecosystems such as RPM, pip, npm or Go modules, or to build them from vendored source. The Codeserver image exposed the difficult cases: its npm graph and installers expected to download binaries during installation. The team replaced those paths with prefetched packages, including multi-architecture Python wheels for ripgrep and Pandoc, and patched post-install behavior to consume the cached binaries.

Conforma checks on the OpenShift AI product path then evaluate hermetic tasks, RPM signatures, multi-architecture consistency, labels and SBOM shape. Exceptions can bridge gaps, but the article argues that they should not become the normal packaging model.

What platform teams can reuse

Teams building custom OpenShift AI workbench images can start with three steps: commit lockfiles, run the shared prefetch process and build with networking disabled. The repository includes generators for RPM, Python, npm and Go dependency sets, plus example Tekton PipelineRuns and local Podman instructions.

The main operational caution is to keep architecture and Python-version inputs aligned with the target image. npm-heavy Codeserver builds also need explicit CPU, memory and disk planning in smaller CI runners.

This pattern is most useful as a release-engineering discipline: make every external input explicit before the build starts, preserve the distinction between upstream and subscribed bases, and fix installers that assume the internet will always be available.