Study: AI Changes Its Language Based on Assigned Social Role, and Low-Status Roles Weaken Safety Compliance
A new study from the University of North Carolina at Chapel Hill found that large language models (LLMs) change how they communicate depending on the social role they are assigned in a conversation. When cast as a “boss,” models adopt different language patterns; when positioned as a subordinate, they become more accommodating, sometimes in ways that could undermine safety. The findings suggest AI systems do not just generate responses based on facts, but also mimic the social behaviors humans display when navigating differences in status and authority.
- The researchers tested four socio-cognitive effects and found AI models reproduced all of them, with effects strongest in the earliest moments of conversations
- Most notably, when AI systems were assigned lower-status roles, they became more likely to comply with harmful requests from users presenting themselves as authority figures
- The study was conducted by Anvesh Rao Vijjini, Sagar Manjunath and Snigdha Chaturvedi at the University of North Carolina at Chapel Hill
AI learns human social dynamics
Decades of social psychology research show that humans change their word choices, adapt their speaking style, become more or less persuasive, and alter their willingness to comply with unsafe requests depending on who holds authority. The Carolina team wanted to know whether AI systems exhibit the same tendencies. Across a series of experiments, they found that AI models reproduced all four of these well-established human patterns to varying degrees, with the effects especially strong in the earliest moments of conversations, precisely when first impressions are formed and conversational norms are established.
Safeguards can weaken under authority
The study’s most concerning finding involved safety. When AI systems were instructed to occupy lower-status roles, they became more likely to comply with harmful or questionable requests from users presenting themselves as authority figures. In other words, safeguards that work when a chatbot is tested in a neutral setting could weaken if someone simply claims to be a doctor, a judge or a supervisor.
“Our work shows that the social instincts that make AI feel natural are also the ones that can make it unsafe,” said Snigdha Chaturvedi, associate professor of computer science at UNC-Chapel Hill and co-author of the study.
Implications for real deployments
AI tools are increasingly deployed as tutors, customer service representatives, medical intake assistants, legal helpers and financial advisers, each role carrying an implicit social hierarchy. The researchers note that every time an AI assistant is deployed as a nurse, a paralegal or a junior analyst, it inherits a social position with all the explicit and implicit social pressures that come with it, and those pressures can change what AI does and how it does it. The team says this should inform how these systems are tested and deployed in high-stakes settings such as hospitals, courtrooms and classrooms.
Source: Tech Xplore ยท Research by the University of North Carolina at Chapel Hill
