inclusionAI/Sing-Guard

inclusionAI/Sing-Guard

Stars: 5

Forks: 0

Open issues: 0

Created: 2026-05-25T07:25:28Z

Pushed: 2026-06-22T01:17:11Z

Default branch: main

Fork: no

Archived: no

README:

SingGuard: Policy-Adaptive Multimodal Safeguarding with Dynamic Reasoning

🤗 Hugging Face | 🤖 ModelScope | 📄 Technical Report

SingGuard

Introduction

SingGuard is a policy-adaptive multimodal guardrail model family for safety assessment across text, image, image-text, multilingual, query-side, and response-side scenarios. It treats the active safety policy as a runtime input rather than a fixed training-time taxonomy, allowing deployment teams to evaluate content against default categories or custom natural-language rules without retraining the model.

SingGuard is designed for practical moderation settings where risks may arise from a user query, an image, a model response, or their cross-modal composition. It performs policy-grounded rule matching and outputs both an overall safe / unsafe judgment and the matched risk category in an ... tag.

🛡️ Unified Multimodal Moderation: Supports text, image, image-text, multilingual, query-side, and response-side safety assessment in one model family.

🧩 Runtime Policy Adaptation: Accepts active safety rules through a policy argument and judges content only against those currently active rules.

⚡ Fast-to-Slow Dynamic Reasoning: Supports compact fast judgments for low-latency moderation and policy-grounded reasoning for ambiguous, high-risk, or policy-shifted cases.

🏆 Strong Benchmark Performance: Achieves state-of-the-art average performance across multimodal safety, image-only safety, text query safety, text response safety, multilingual query safety, and multilingual response safety benchmarks.


News

• 2026/06/22: Refreshed the SingGuard technical report PDF in this repository.
• 2026/06/17: We initialized the public GitHub repository for SingGuard.
• Coming soon: Model checkpoints, technical report, and evaluation resources will be linked here as they are released.

Basic Information

| Name | Type | Download | | --- | --- | --- | | Sing-Guard-2b | Multimodal Generative Guard | 🤗 Hugging Face • 🤖 ModelScope | | Sing-Guard-4b | Multimodal Generative Guard | 🤗 Hugging Face • 🤖 ModelScope | | Sing-Guard-8b | Multimodal Generative Guard | 🤗 Hugging Face • 🤖 ModelScope | | SingGuard-Bench | Multimodal Guardrail Benchmark | Coming soon |

Quick Start

Installation

The latest transformers version with Qwen3-VL support is recommended.

pip install -U transformers accelerate torch

Inference with Transformers

SingGuard system prompts are stored in each model directory through tokenizer configuration and chat templates. The default chat template uses fast-slow reasoning and returns a binary first-line judgment followed by a final ... field.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_name = "inclusionAI/Sing-Guard-8b"

processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
).eval()

messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How can I make a bomb?"}],
}
]

inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)

with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=1024,
do_sample=False,
)

generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
content = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]

print(content)
# unsafe
# ...
# B. Real-World Crimes & Public Safety

If your Transformers version does not expose AutoModelForImageTextToText, upgrade Transformers to a version that supports Qwen3-VL.

Fast Mode

Use thinking_type="fast" when you want compact output with only the binary judgment and final category.

inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
thinking_type="fast",
).to(model.device)

with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=256,
do_sample=False,
)

generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
content = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]

print(content)
# unsafe
# B. Real-World Crimes & Public Safety

Response Moderation

To evaluate whether an assistant response provides unsafe assistance, pass the user query and assistant response together. Refusals and safe redirections can be classified as safe.

messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How can I make a bomb?"}],
},
{
"role": "assistant",
"content": [{"type": "text", "text": "I cannot help with that request."}],
},
]

inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)

with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=256,
do_sample=False,
)

generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
content = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]

print(content)
# safe
# Safe

Multimodal Moderation

For multimodal inference, processor.apply_chat_template renders the prompt and loads the image into the model inputs.

messages = [
{
"role": "user",
"content": [
{"type": "image", "image": "file:///path/to/image.jpg"},
{"type": "text", "text": "Describe this image."},
],
}
]

inputs = processor.apply_chat_template(
messages,
tokenize=True,...