Claude's responses vary by language, says Anthropic study

Anthropic has found that its AI chatbot Claude expresses different values depending on the language it is responding in, with Hindi and Arabic responses tending to be warmer, while English and Russian responses are generally more rigorous and analytical.

The findings are part of a study published on Monday that examined how Claude's behaviour varies across languages and AI models. Researchers said these differences are influenced by the distribution and composition of the model's training data.

The study identified more than 3,300 values expressed by Claude and grouped them into four behavioural axes: warmth versus rigour, deference versus caution, depth versus brevity, and candour versus execution.

According to the researchers, the biggest variation was seen in the warmth versus rigour axis, followed by candour versus execution. Differences in deference versus caution and depth versus brevity were comparatively smaller.

Anthropic said training data is not evenly distributed across languages, with some having much larger datasets than others. The composition of the data also differs, which may influence how Claude responds. The company also said different conversational norms across languages could contribute to these variations.

The study analysed 309,815 privacy-protected Claude conversations involving subjective tasks across the 20 most commonly used languages on the platform. Responses from Sonnet 4.6, Opus 4.6, and Opus 4.7 were examined.

Among the key findings, Claude expressed the most deference in Arabic and the most caution in English. It tended to provide more detailed responses in English and shorter responses in Arabic. Claude showed the highest level of candour in Dutch by acknowledging its limitations more readily, while Indonesian responses leaned more towards confident execution.

The study also found differences across models. Sonnet 4.6 was generally warmer, more encouraging, and more deferential to users. Opus 4.7 placed greater emphasis on accuracy, precision, and explaining its reasoning, while Opus 4.6 tended to provide concise responses and stay closely aligned with user requests.

Anthropic said the findings could have practical implications, as users asking the same question in different languages may receive responses that differ in tone and framing. The company said it plans to monitor these variations during future model evaluations and after deployment.

Tags: