Atos tunes small models on OpenShift AI for structured German-language analysis

Atos has described a public-sector proof of value in which small language models fine-tuned on Red Hat OpenShift AI produced more reliable structured output than a GPT-4.1 baseline on a narrowly defined German-language task.

The results come from an Atos and Red Hat engineering account published Oct. 27. They are not a general model comparison: the team built a custom evaluation around one customer’s requirements, including extraction of 14 features from German public social-media comments, valid JSON generation and detection of calls for violence.

Synthetic data and modest training hardware

The team used GPT-4.1 as a teacher model to generate synthetic training data, then fine-tuned Phi-4 Mini 3.8B, Qwen 2.5 1.5B and Qwen 2.5 7B student models. OpenShift AI workbenches and Jupyter notebooks provided the development environment, while training jobs ran through KFTrainer and the Kubeflow SDK.

According to the published account, data generation used an external model API and no local GPU. Fine-tuning used three Nvidia L40 GPUs. The project tested supervised fine-tuning and orthogonal subspace fine-tuning across multiple epochs.

The evaluation covered 27 tuned model variations and 12 reference configurations, for 39 runs in total. Its 34-point rubric allocated 28 points to extraction quality, three to valid and complete JSON, and three to violence classification against a human-validated reference.

Results need a narrow reading

The strongest reported improvement was output reliability. The article says the untuned Phi-4 Mini generated valid JSON only 1% of the time; after one epoch that rose to 92%, and further training brought it to about 99%. GPT-4.1 reached 96% in the same evaluation.

On the overall rubric, GPT-4.1 scored about 29 points with a short prompt and 30.5 with a detailed prompt. The tuned Phi-4 Mini reached about 30 points, while tuned Qwen 2.5 7B reached 30.8. The team also reported that even the 1.5B-class Qwen model performed near perfectly on JSON formatting, entity extraction and violence detection.

Those figures support a specific conclusion rather than a universal one: for a tightly scoped workflow with a domain-specific dataset and scoring method, model customization can matter more than parameter count. The case study does not establish that the smaller models outperform frontier models across general reasoning or unrelated tasks.

For regulated teams, the operational attraction is clear. The resulting student models can be self-hosted, while the training experiment used infrastructure small enough for iterative work. The unresolved question is reproducibility: the article describes the methodology and scores, but does not publish the customer dataset or a directly runnable evaluation package.