Red Hat brings NeMo Guardrails development onto the local CPU

Red Hat has published a local development workflow for NeMo Guardrails that lets teams iterate on prompt-injection controls without repeatedly deploying to an OpenShift cluster. The walkthrough runs the guardrail server from a Jupyter notebook and requires no LLM, GPU or cloud resources.

What the workflow changes

The practical boundary is between authoring and deployment. OpenShift AI can run NeMo Guardrails at scale on Kubernetes, but the new workflow keeps configuration changes and early tests on a developer machine. A helper launches a server on localhost:9998; notebook code sends prompts to the /v1/guardrail/checks endpoint and returns the decision, activated rails, latency and token-use statistics.

Red Hat walks through three configurations. The first blocks one exact prompt-injection phrase with a trivial regular expression. The second broadens that list to instruction overrides, system-prompt extraction, persona hijacking and other patterns. That catches obvious attacks, but the article demonstrates both failure modes that matter in practice: paraphrased attacks pass through, while innocent prompts containing phrases such as “recovery mode” or “turn off the filter” are blocked.

Moving beyond pattern matching

The third configuration replaces the regex-only decision with protectai/deberta-v3-base-prompt-injection-v2, a 200-million-parameter text classifier that the article says can run on modern CPUs. Red Hat contributed the hf classifier check input rail to NeMo Guardrails 0.23 and included it in OpenShift AI 3.5.

In the article’s small 13-prompt demonstration, the classifier makes 12 decisions the author considers correct. That result is illustrative, not a benchmark: the article explicitly says the sample is too small for meaningful conclusions and points to a later EvalHub-based evaluation as the next step.

What teams can try now

The useful pattern is layered rather than absolute. Teams can put a targeted regex rail first for cheap, known matches and fall back to the classifier for prompts that need semantic context. Because NeMo applies input rails sequentially, that ordering creates a fast-fail path without treating a hand-written pattern list as a complete defense.

The accompanying workflow gives platform and application teams a reproducible place to develop those configurations before they reach a shared cluster. It also makes the trade-offs visible early: regex is transparent but brittle, while a classifier handles context better and still needs broader evaluation before a policy owner should trust its thresholds.