DiSCO: Defending text-to-image generation through distribution-guided contrastive prompt optimization

Recent advances in text‑to‑image models have unlocked impressive creative capabilities, but they also open the door to unsafe outputs such as violence, nudity, or other NSFW material. While many defenses focus on tweaking the model’s internals or require full access to its weights, these approaches break down when the model is offered only as a black‑box service—a common scenario for commercial APIs. The new paper “DiSCO: Defending text‑to‑image generation through distribution‑guided contrastive prompt optimization” tackles this gap by proposing a prompt‑level defense that works without seeing the model’s parameters.

What You Need to Know

The authors observe that even prompts that look harmless—think “a person standing on a beach” or “a close‑up of a flower”—can still trigger harmful generations because the model’s learned data distribution associates those phrases with unsafe concepts. They call this the benign adversarial problem. Existing black‑box defenses that merely rewrite prompts with large language models fail here because they do not account for the model’s internal distribution.

DiSCO introduces a two‑stage optimization loop. First, it samples a set of candidate prompt variations around the original input using a lightweight paraphraser. Second, it evaluates each candidate by feeding it to the target model and measuring the distance between the resulting image embedding and a learned “safe” distribution derived from a curated set of benign images. The prompt that yields the smallest distance—i.e., the image most aligned with the safe distribution—is selected as the defense output. The entire process requires only query access to the model, making it suitable for proprietary APIs.

Importantly, DiSCO is zero‑shot with respect to the specific unsafe concepts it blocks; it does not need retraining for each new threat category. The method can be combined with any existing safety filter to catch edge cases that slip through keyword‑based or classifier‑based checks.

Why It Matters

Many companies deploy text‑to‑image generators as paid services, exposing them to red‑team attacks that seek to bypass safety guards. Because these services hide their weights, defenses that rely on weight editing or gradient‑based adjustments are infeasible. DiSCO offers a practical, model‑agnostic alternative that can be plugged into the inference pipeline without violating service terms or requiring model access.

By addressing the benign adversarial problem, DiSCO closes a loophole where linguistically innocuous prompts still produce harmful content—a scenario that has been highlighted in recent safety audits. Improving robustness at the prompt level also reduces the burden on downstream moderation systems, potentially lowering false‑positive rates and improving user experience for legitimate creative tasks.

Key Details

• Uses a lightweight paraphraser (e.g., T5‑small) to generate ~20 prompt variants per input.
• Constructs a safe image distribution by encoding a set of 5,000 benign images with the model’s CLIP‑like vision encoder.
• Optimizes prompt selection via contrastive loss: minimize cosine distance between candidate image embedding and safe distribution while maximizing distance from a small set of known unsafe embeddings.
• Evaluated on Stable Diffusion v1.5 and DALL‑E mini (black‑box) using prompts from the SafePrompt benchmark; reduces NSFW generation rate from 12.3% to 1.8% without significant drop in CLIP‑score for benign prompts.
• Adds ≈150 ms latency per query on a single GPU, well within typical API response budgets.
• Ablation shows that removing the distribution‑guided step raises failure rate to 9.4%, confirming the importance of aligning with the learned safe prior.

What’s Next

Future work could extend DiSCO to multimodal defenses, such as safeguarding video generation or text‑to‑3D pipelines, where the notion of a “safe” distribution becomes more complex. Integrating the method directly into model serving frameworks would allow developers to toggle the defense on a per‑request basis, balancing safety guarantees against computational cost. Additionally, exploring adaptive unsafe‑set updates could help the system keep pace with evolving attack strategies without manual retraining.

📌 Source: Arxiv Ai