Can AI explain human behavior? Using large language models to map human motivations

Can artificial intelligence help us better understand ourselves? 

New research by University of Michigan School of Information PhD candidate Yutong Xie and UMSI professor Qiaozhu Mei explores how large language models (LLMs) can reveal the hidden motivations behind human decision-making in classic economic games like the Dictator Game and Bomb Risk Game. 

Published in the Proceedings of the National Academy of Sciences (PNAS), “Using large language models to categorize strategic situations and decipher motivations behind human behaviors” is cowritten by UMSI PhD candidate Yutong Xie and professor and associate dean Qiaozhu Mei along with Walter Yuan (MobLab) and Matthew O. Jackson (Stanford University). 

The study introduces the concept of “behavioral codes” which prompt artificial intelligence models to mimic different strategies such as generosity, self-interest, or cooperation. These behavioral codes allow researchers to see how AI can replicate a wide range of human choices. 

“In strategic games and everyday interactions, we observe each other’s behaviors,” Mei says. “But if you ask people why they made a certain decision, they’re often not very good at explaining it. Sometimes people even make up stories that sound good but don’t reflect their true motivations. With LLMs, we have a new way of probing behavior. Like the human mind, LLMs are black boxes, but unlike people, we can systematically poke them by creating system prompts.’”

The study focused on widely used behavioural economic games like Dictator Game, Ultimatum Game and Prisoner’s Dilemma. For this paper, researchers worked with ModLab, which built these games as classroom teaching tools and has, for decades, collected large datasets on human behaviour the researchers analyzed. 

“These games have been tested across many different populations for years,” Mei says. “That gave us the data we needed to derive and validate behavioral codes.” 

The findings from this paper show that AI can be used to compare how different groups of people, like students from various backgrounds, make decisions, offering new insight into the motivations that shape their behavior. 

“In education, for example, we could customize teaching approaches based on student motivations,” Mei says. “In the workplace, we could design incentive systems that better align with a worker’s behavioral traits.” 

The paper is part of Mei’s broader research goals of combining AI and behavioral science. Alongside colleagues at Standford, Mei is helping define a new field — AI behavioral science — with two goals: helping advance AI to study human behavior, and using behavioral science to better understand AI itself. 

In the future, Mei says, he would like to emphasize the importance of collaboration between humans and machines. 

“We don’t see AI replacing humans,” he said. “We’re aiming for a world where humans and AI are greater together.” 

Read “Using large language models to categorize strategic situations and decipher motivations behind human behaviors” in PNAS and see the abstract below: 

By varying prompts to a large language model, we can elicit the full range of human behaviors in a variety of different scenarios in classic economic games. By analyzing which prompts elicit which behaviors, we can categorize and compare different strategic situations, which can also help provide insight into what different economic scenarios might induce people to think about. We discuss how this provides a step toward a nonstandard method of inferring (deciphering) the motivations behind the human behaviors. We also show how this deciphering process can be used to categorize differences in the behavioral tendencies of different populations.

RELATED 

Qiaozhu Mei is a professor and associate dean for research and innovation at UMSI, and a professor of electrical engineering and computer Science at the College of Engineering. Learn more about him by visiting his UMSI faculty profile. 

Yutong Xie is a PhD candidate at UMSI. Learn more about her by visiting her UMSI profile.