We present SGuard-v1, a lightweight safety guardrail for Large Language Models (LLMs), which comprises two specialized models to detect harmful content and screen adversarial prompts in human-AI conversational settings. The first component, ContentFilter, is trained to identify safety risks in LLM prompts and responses in accordance with the MLCommons hazard taxonomy, a comprehensive framework for trust and safety assessment of AI. The second component, JailbreakFilter, is trained with a carefully designed curriculum over integrated datasets and findings from prior work on adversarial prompting, covering 60 major attack types while mitigating false-unsafe classification. SGuard-v1 is built on the 2B-parameter Granite-3.3-2B-Instruct model that supports 12 languages. We curate approximately 1.4 million training instances from both collected and synthesized data and perform instruction tuning on the base model, distributing the curated data across the two component according to their designated functions. Through extensive evaluation on public and proprietary safety benchmarks, SGuard-v1 achieves state-of-the-art safety performance while remaining lightweight, thereby reducing deployment overhead. SGuard-v1 also improves interpretability for downstream use by providing multi-class safety predictions and their binary confidence scores. We release the SGuard-v1 under the Apache-2.0 License to enable further research and practical deployment in AI safety.

翻译：本文提出SGuard-v1，一种用于大型语言模型（LLMs）的轻量级安全护栏，该护栏包含两个专用模型，用于检测人机对话场景中的有害内容并筛选对抗性提示。第一个组件ContentFilter经过训练，可根据MLCommons危害分类法（一个全面的AI信任与安全评估框架）识别LLM提示和响应中的安全风险。第二个组件JailbreakFilter通过精心设计的课程在集成数据集上进行训练，并借鉴了先前对抗性提示研究中的发现，覆盖了60种主要攻击类型，同时减少了误判为不安全的情况。SGuard-v1基于支持12种语言的20亿参数Granite-3.3-2B-Instruct模型构建。我们从收集和合成的数据中整理了约140万个训练实例，并在基础模型上进行了指令微调，根据两个组件的指定功能分配整理的数据。通过在公开和专有安全基准上的广泛评估，SGuard-v1实现了最先进的安全性能，同时保持轻量化，从而降低了部署开销。SGuard-v1还通过提供多类安全预测及其二元置信度分数，提高了下游应用的解释性。我们根据Apache-2.0许可证发布SGuard-v1，以促进AI安全领域的进一步研究和实际部署。