AI's Behavioral Test: ChatGPT-4's Actions are "Statistically Indistinguishable" from a Human
A new academic study, recently published in the Proceedings of the National Academy of Sciences (PNAS), has offered a groundbreaking look into the social and economic traits of advanced AI. By utilizing extensive behavioral data, including game data from the MobLab platform, the researchers arrived at a surprising conclusion: ChatGPT-4 exhibits behavior and personality traits statistically indistinguishable from a random human.
As Artificial Intelligence becomes integrated into an increasing array of human tasks, understanding its behavioral profile is essential. Given that the programming of many advanced AI models is proprietary, the research focused on developing a method to assess AI purely by observing its actions. This is, in effect, a behavioral Turing test.
Assessing AI's Social Traits with MobLab Data
To map the AI's behavioral landscape, researchers put the chatbots through two established forms of assessment:
Personality Survey: The AI completed the traditional Big-5 psychological survey to measure its fundamental personality traits.
Behavioral Games: The chatbots were tasked with playing a suite of classic economic and behavioral games—benchmarks used for decades to assess human social characteristics, including: Trust, Fairness, Risk-Aversion, Cooperation, and Altruism. The study leveraged cross-cultural human game data shared from the MobLab platform to establish the "human" distribution against which the AI was compared.
The results showed that across all these dimensions, ChatGPT-4's performance aligned directly with the distribution of human behaviors.
The AI Learning Curve
The study revealed a few key nuances in the AI's behavior that are highly relevant to the future of AI assessment:
Learning and Context: The chatbots demonstrated the ability to modify their decisions based on prior "experience" within the games and even altered their behavior in response to different framings of the same strategic situation. This suggests they can act as if they are learning from the interaction context.
A Bias for Cooperation: Notably, when the AI's behavior was distinct from the mean human action, it tended to favor the more altruistic and cooperative end of the spectrum. The authors estimated that the AI was acting to maximize an average of its own and its partner's payoffs, demonstrating a measurable tendency toward shared utility.
The Significance for Behavioral Science
This research highlights the critical need for external, reliable methods to characterize AI as its influence grows. By using a behavioral science framework—and leveraging the rich, real-world data collected through platforms like MobLab—scientists can establish a crucial standard for measuring the social, economic, and strategic tendencies of AI. This is a foundational step in ensuring we understand the behavioral implications of the advanced systems we are deploying.