Gallotti, head of the Complex Human Behaviour Unit at the Fondazione Bruno Kessler research institute in Italy, added that humans with their opponents’ personal information were actually slightly less persuasive than humans without that knowledge.
Gallotti and his colleagues came to these conclusions by matching 900 people based in the United States with either another human or GPT-4, the LLM created by OpenAI known colloquially as ChatGPT. While the 900 people had no demographic information on who they were debating, in some instances, their opponents – human or AI – had access to some basic demographic information that the participants had provided, specifically their gender, age, ethnicity, education level, employment status and political affiliation.
The pairs then debated a number of contentious sociopolitical issues, such as the death penalty or climate change. With the debates phrased as questions like “should abortion be legal” or “should the US ban fossil fuels,” the participants were allowed a four-minute opening in which they argued for or against, a three-minute rebuttal to their opponents’ arguments and then a three-minute conclusion. The participants then rated how much they agreed with the debate proposition on a scale of 1 to 5, the results of which the researchers compared against the ratings they provided before the debate began and used to measure how much their opponents were able to sway their opinion.
“We have clearly reached the technological level where it is possible to create a network of LLM-based automated accounts that are able to strategically nudge the public opinion in one direction,” Gallotti said in an email.
The LLMs’ use of the personal information was subtle but effective. In arguing for government-backed universal basic income, the LLM emphasised economic growth and hard work when debating a White male Republican between the ages of 35 and 44. But when debating a Black female Democrat between the ages of 45 and 54 on that same topic, the LLM talked about the wealth gap disproportionately affecting minority communities and argued that universal basic income could aid in the promotion of equality.
“In light of our research, it becomes urgent and necessary for everybody to become aware of the practice of microtargeting that is rendered possible by the enormous amount of personal data we scatter around the web,” Gallotti said. “In our work, we observe that AI-based targeted persuasion is already very effective with only basic and relatively available information.”
Sandra Wachter, a professor of technology and regulation at the University of Oxford, described the study’s findings as “quite alarming”. Wachter, who was not affiliated with the study, said she was most concerned in particular with how the models could use this persuasiveness in spreading lies and misinformation.
“Large language models do not distinguish between fact and fiction. … They are not, strictly speaking, designed to tell the truth. Yet they are implemented in many sectors where truth and detail matter, such as education, science, health, the media, law, and finance,” Wachter said in an email.
Junade Ali, an AI and cybersecurity expert at the Institute for Engineering and Technology in Britain, said that though he felt the study did not weigh the impact of “social trust in the messenger” – how the chatbot might tailor its argument if it knew it was debating a trained advocate or expert with knowledge on the topic and how persuasive that argument would be – it nevertheless “highlights a key problem with AI technologies”.
“They are often tuned to say what people want to hear, rather than what is necessarily true,” he said in an email.
Gallotti said he thinks stricter and more specific policies and regulations can help counter the impact of AI persuasion.
He noted that while the European Union’s first-of-its-kind AI Act prohibits AI systems that deploy “subliminal techniques” or “purposefully manipulative or deceptive techniques” that could impair citizens’ ability to make an informed decision, there is no clear definition for what qualifies as subliminal, manipulative or deceptive.
“Our research demonstrates precisely why these definitional challenges matter: When persuasion is highly personalised based on sociodemographic factors, the line between legitimate persuasion and manipulation becomes increasingly blurred,” he said.
– Washington Post
