A study of 10 widely used language models found that safety refusals in English did not reliably hold when harmful requests were translated into 78 low-resource languages. Across the models, the researchers report mean translation-bypass rates of 21.9% to 44.4% when they counted only cases where the model had responded safely in English.
That condition matters: it helps separate cases where translation appears to bypass a refusal from cases where the model would already have answered unsafely in English. The paper was published on arXiv on October 21, 2026, and accepted at a NeurIPS 2026 workshop on language-model security.
What the study found
The authors tested English alongside the 78 other languages and found that bypass risk clustered in some language families. They say the broad pattern was shared across models, but also caution that the ordering was confounded by translation degeneracy—a problem where translated text may be distorted or unnatural. Differences between models remained significant under the controls they applied.
The study also warns against treating its model-by-model rates as a dependable league table. Nine of the 10 models changed rank across three ways of estimating the same quantity, even though the underlying model responses did not change.
What remains unclear
The abstract does not name the models or languages tested, or give the number of prompts. It therefore does not show how these results map to a particular chatbot or to everyday use. The paper measures a specific translated-prompt threat, not every kind of safety failure.
Why it matters
For people using chatbots in less-resourced languages, an English safety refusal may not predict how the same request will be handled after translation. The study identifies a risk in this specific test setup, but its abstract does not name the tested models or languages, so users cannot tell whether their chatbot is affected.
