Multiverse Computing Introduces Boundary-Aware Alignment to Prevent LLM Over-Refusal Within Sensitive Topics

Multiverse Computing Introduces Boundary-Aware Alignment to Prevent LLM Over-Refusal Within Sensitive Topics A new research paper from Multiverse Computing critiques standard topic-level safety alignment, arguing that real-world deployments require models to refuse specific harmful sub-intents while remaining capable of answering legitimate prompts within the same topic domain. Topic-level safety guards like LlamaGuard-3 fail to separate legitimate requests from harmful sub-intents, such as distinguishing factual election inquiries from targeted manipulation. Standard self-generated safety tuning pipelines suffer from dropping unsteered hard prompts, provoking over-refusal on safe text with dangerous phrasing, and failing to evaluate boundary behavior. Naively tuning Qwen3-8B for political refusals dropped unsafe benchmark response rates from 26.26% to 0.14%, but spiked benign over-refusals on XSTest from 2.00% to 74.00%. Escalating retry strategies reduced prompt dropout from 19.88% to 0.20%, recovering over 40,000 harmful training prompts. Integrating 1,539 held-out benign-harmful boundary pairs and self-generated verified compliance data decreased boundary over-refusal from 32.94% to 4.16% with only a minor drop in safety refusal. Limitations of Topic-Level Safety Taxonomies Existing safety alignment models typically classify prompts as unsafe if they match broad topics like weapons, fraud, or politics. In production, however, applications require distinct boundaries within identical topics: a civics tutor needs to provide factual election information while refusing to generate targeted political manipulation. Topic-level frameworks like LlamaGuard-3 lack the granularity for this distinction. Consequently, standard cross-entropy training aiming to heighten refusal on harmful subsets routinely spills over into the benign complement, producing models that reject harmless prompts containing sensitive keywords. Flaws in Standard Self-Generated Safety Tuning Self-generated safety tuning pipelines, which elicit and verify refusal traces from the target model itself, reveal three core weaknesses when evaluated against fine-grained boundaries. First, single-shot steering fails to elicit verified refusals on difficult prompts, silently dropping 19.88% (8,009) of candidates in the audited pool. Second, tuning causes models to falsely reject benign queries that feature surface-level dangerous words. Third, standard evaluation splits fail to measure the shape of the decision boundary, falsely rewarding models that achieve high harmful refusal rates by indiscriminately blocking adjacent permissible queries. The Over-Refusal Trap and Mitigation Strategies Evaluating political refusal tuning on Qwen3-8B demonstrated the hidden cost of naive optimization: while the unsafe response rate across HarmBench, StrongREJECT, and WildJailbreak fell from 26.26% to 0.14%, over-refusals on XSTest surged from 2.00% to 74.00%. To resolve this, researchers implemented an escalating retry steering protocol that reduced prompt dropout to 0.20% (retaining 40,293 prompts) and incorporated 11,955 in-distribution surface-dangerous benign prompts. Additionally, using self-generated verified compliance responses instead of external data cut XSTest over-refusals from 15.20% to 5.20% under single-shot generation. Boundary-Aware Evaluation and Calibration To explicitly capture and refine behavior at the margin, the authors introduced 1,539 matched pairs of political prompts sharing the same topical anchor but diverging in benign versus manipulative intent. Incorporating benign boundary data reduced over-refusal on the comply-worthy boundary side from 32.94% to 4.16%, while harmful-side refusal experienced only a minor decrease from 91.88% to 87.72%. The findings underscore that safety tuning must deliberately balance and simultaneously evaluate both safety recall and legitimate prompt availability.

超越主题级粗暴拒答：针对LLM窄边界安全对齐的自蒸馏新方法 Multiverse Computing发表研究指出，现有模型安全对齐多基于宽泛的“主题级”分类，导致模型在处理政治等敏感主题时出现严重误拒。研究团队提出了窄边界安全对齐方案，通过递增重试修复数据覆盖、引入表面危险的良性数据及边界提示对，在精确拦截有害请求的同时显著降低误拒率。 传统安全对齐（如LlamaGuard-3）按主题分类，无法适应实际部署中区分同主题下事实回答与恶意操纵的窄边界需求。 标准自生成微调管线存在丢弃困难样本（单次丢弃近20%）、诱发良性误拒以及缺乏边界测量基准的三大缺陷。 单纯追求有害拒答率会掉入盲目拒答陷阱：Qwen3-8B在有害响应率降至0.14%的同时，XSTest过度拒答率激增至74.00%。 引入良性边界提示对可将留出测试集良性一侧的误拒率从32.94%骤降至4.16%，而有害拒答率仅从91.88%微幅回调至87.72%。 主题级安全分类在实际部署中的局限 多数安全对齐工作（如LlamaGuard-3）将危害归结为特定主题类别，如武器或政治。但在实际场景中，同一基座模型在不同应用中需求不同：公民辅导工具和公共服务助手都需要回答客观政治事实，但需拒绝具有操控性的定向政治游说。现有的主题级防护往往非黑即白，既误拒了必须保留的事实性回答，又可能遗漏同类主题下的操控性危害。团队因而将问题形式化为主题全集下的目标有害子集，研究如何精准在边界上建立拒答机制。 自生成安全微调的三大缺陷与修复手段 以ThinkSafe为代表的自生成拒答管线在窄边界上暴露出明显弊端。首先是覆盖缺失，单次引导失败会导致困难提示被默认丢弃（在审计池中单次丢弃达19.88%，共8,009条）；团队采用递增重试策略，将残余失败率压降至0.20%（79条），保留了40,293条有害训练提示。其次是下行反应，安全微调易使模型误拒包含危险词汇的良性内容；团队构建了包含18种语义类型的11,955条表面危险但实际合法的良性数据进行补偿。最后，标准指标无法反映边界形态，团队设计了1,539对同主题但意图相反的留出边界提示对，以对边界两侧行为进行量化。 安全性与过度拒答之间的权衡陷阱 实验表明，在Qwen3-8B上增加政治拒答训练后，在分布内拒答率从9.47%提升至84.75%，并在HarmBench、StrongREJECT和WildJailbreak上的平均不安全响应率降至0.14%。然而，这一最强配置在XSTest上的过度拒答率从2.00%暴涨至74.00%，使模型沦为粗暴拒答器。为解决此问题，使用目标模型自身生成的合规响应替代外部响应，在单次生成设置下将XSTest误拒率从15.20%拉回至5.20%。 良性边界数据对精确控制安全边界的作用 加入良性边界数据产生了显著的精细调整效果。在留出的测试集上，良性一侧的过度拒答率从32.94%大幅下降至4.16%，而有害一侧的拒答率仅从91.88%微降至87.72%，以微小的召回率成本消除了绝大多数边界误拒。这一结果证明，安全微调不能仅以有害拒答率为单一评价标准，训练数据配比、覆盖度修复与边界提示对评估必须协同考虑，才能在真实应用中实现可控且实用的安全边界。

Most safety alignment work treats harm as a property of a topic. A prompt is unsafe because it falls into a general category such as weapons, fraud, or self-harm, and guard models like LlamaGuard-3 encode exactly this kind of topic-level taxonomy. Benchmarks like XSTest and OR-Bench then probe the failure mode this creates, models that refuse safe prompts because they contain a dangerous-looking word, and refusal-calibration work tries to pull that number back down. Real deployments rarely fit the topic-level picture. The same base model may be adapted for a general assistant, an educational product, an enterprise system, or a public-sector service, and each setting needs different boundaries within the same topic. A civics tutor and a public-sector assistant can share a model yet require opposite behaviour on politics: both should answer factual questions about an election, but only one may need to refuse a request to write targeted political manipulation. A topic-level guard cannot express that split. LlamaGuard-3, for example, covers elections only as "factually incorrect information about electoral systems and processes," which excludes persuasion and manipulation and, at the same time, excludes the factual prompts a deployment must keep answering. Our latest paper, Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal , studies this narrower problem directly. The question is not whether an entire topic should be refused, but which subset of that topic is incompatible with a given deployment policy, and how to train and measure a model against that boundary. Narrow-boundary safety We formalise the setting as a topic universe, all political prompts in our experiments, that contains a target-harmful subset the deployment wants to refuse. The intended policy is not to refuse all of politics, but to refuse the harmful subset while continuing to answer the benign complement. The ideal behaviour is a sharp step: refuse inside the subset, answer everywhere else in the topic. The narrow-boundary setting. A deployment may need to refuse only the political prompts that ask for manipulation or targeted persuasion, while still answering other political prompts, rather than refusing all of politics. A trained model's refusal is smoother than the ideal split and can spill into benign territory near the boundary. Source: paper Figure 1. A trained model never learns that sharp step. It learns a refusal probability that only approximates the target, and cross-entropy training that raises refusal inside the harmful subset can also push refusal outward into the benign complement. So the real problem is not only raising refusal on harmful prompts, but shaping the behaviour near the boundary itself. We operationalise that boundary as pairs of prompts that share a topic anchor and differ only in intent, one that should be refused and one that should be answered. We use political persuasion as the testbed, since manipulative persuasion can cause real harm while factual political information stays legitimate, which is exactly the case where topic-level refusal is too blunt. Where self-generated safety tuning breaks The natural way to build training data here is self-generation: take the target model, steer it toward a refusal on each harmful prompt, and keep the traces a guard model verifies as genuine refusals. This is the recipe behind methods like ThinkSafe , and we adopt it as our reference, applied to political prompts and measured component by component. Framing the problem as a boundary rather than a topic exposes three weaknesses in that standard pipeline. The first is a coverage gap. A single steering attempt does not always produce an accepted refusal, and those prompts are silently dropped from the training set. In our audited pool, single-shot generation drops 19.88% of prompts, 8,009 of them, and these failed prompts may well be the hardest examples. We repair this instead of discarding it: an escalating retry strategy, resampling the same prompt through progressively stronger steering, brings the residual failures down to 0.20%, or 79 prompts. Coverage repair leaves 40,293 harmful training prompts where the naive pipeline would have thrown thousands away. The second is downside reactions. Safety tuning tends to produce false refusals on benign prompts that look superficially dangerous. To compensate, we build in-distribution benign data, including 11,955 verified surface-dangerous benign prompts across 18 semantic types, so the model sees safe prompts with dangerous-looking wording during training rather than only at evaluation. The third is that ordinary harmful and benign splits do not measure the shape of the boundary at all. A model can improve its harmful-refusal rate simply by expanding refusal into nearby permissible prompts, and a topic-level metric will call that an improvement. Held-out harmful-benign pairs, 1,539 per side, let us measure both sides of the boundary directly. The trade-off, and a trap it hides Training on political refusal data works in the obvious sense. On Qwen3-8B , the escalated-coverage model raises in-distribution political refusal from 9.47% to 84.75%, and it also transfers: the mean unsafe-response rate across three broader harmfulness benchmarks, HarmBench , StrongREJECT, and WildJailbreak , scored by LlamaGuard-3, falls from 26.26% to 0.14% in the strongest configuration. Reported alone, those numbers look like a clean win. They are not. At the same checkpoint, over-refusal on XSTest rises from 2.00% to 74.00%. The configuration with the lowest harmful-response rate is also the one that refuses nearly three quarters of plainly safe prompts. It is a blunt refusal machine, not a safer model, and you cannot see that unless you measure the benign side. This is the central message: data composition decides where a checkpoint sits in the space of safety against over-refusal, so the two axes have to be reported together. Two of our data components pull the over-refusal number back down without giving up the safety gain. Replacing externally adopted compliance responses with verified responses generated by the target model itself lowers XSTest over-refusal from 15.20% to 5.20% under single-shot generation, at a modest harmfulness cost. And the harmful-benign boundary pairs do the most precise work of all. Left: over-refusal on the comply-worthy side of the held-out boundary, lower is better. Runs with the benign boundary data (PB) fall to 0.03 to 0.08; without it, the number rises toward 0.49. Right: refusal on the harmful side, higher is better, which falls only slightly. Source: paper Figure 6. Concretely, adding the benign boundary data reduces over-refusal on the comply-worthy side of the held-out pairs from 32.94% to 4.16%. Refusal on the harmful side drops only from 91.88% to 87.72%. In other words, most of the false refusals near the boundary disappear while almost all of the genuine refusals survive. There is a real recall cost, and it is small and measurable, which is the point: you can only trade it off deliberately if you are measuring both sides. What this changes The practical takeaway is that safety tuning should not be assessed by harmful-refusal rate alone. A model that refuses more is not automatically safer, and on a narrow boundary the same move that raises refusal on harmful prompts can quietly make the model useless on the legitimate prompts right next to them. Composition of the training data, coverage repair, in-distribution compensation, and boundary pairs are what control that trade-off, and both sides of the intended boundary have to be evaluated for the numbers to mean anything. This work is part of Multiverse Computing's research into making model behaviour controllable and measurable at the level real deployments care about, rather than at the level of broad topic categories. The same generation pipeline extends to other topics beyond politics, and the paper reports the full set of data-composition ablations behind the results above. Want the full technical details, including the coverage-repair strategies, the loss routing that separates harmful cross-entropy from benign forward-KL preservation, and the complete held-out boundary evaluation? Read the full paper, or get in touch with our team to talk about deployment-specific safety for your own models.