Generative Agent Simulations of 1,000 People

id:

2411.10109

Authors:

Joon Sung Park, Carolyn Q. Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, Michael S. Bernstein

Published:

2024-11-15

arXiv:

https://arxiv.org/abs/2411.10109

PDF:

https://arxiv.org/pdf/2411.10109

DOI:

N/A

Journal Reference:

N/A

Primary Category:

cs.AI

Categories:

cs.AI, cs.HC, cs.LG

Comment:

N/A

github_url:

_

abstract

The promise of human behavioral simulation–general-purpose computational agents that replicate human behavior across domains–could enable broad applications in policymaking and social science. We present a novel agent architecture that simulates the attitudes and behaviors of 1,052 real individuals–applying large language models to qualitative interviews about their lives, then measuring how well these agents replicate the attitudes and behaviors of the individuals that they represent. The generative agents replicate participants’ responses on the General Social Survey 85% as accurately as participants replicate their own answers two weeks later, and perform comparably in predicting personality traits and outcomes in experimental replications. Our architecture reduces accuracy biases across racial and ideological groups compared to agents given demographic descriptions. This work provides a foundation for new tools that can help investigate individual and collective behavior.

premise

outline

quotes

notes

summary

  1. Brief Overview

This paper introduces a novel agent architecture that simulates the attitudes and behaviors of over 1,000 real individuals using large language models and two-hour qualitative interviews. The accuracy of the generated agents is benchmarked against established social science measures, demonstrating high predictive accuracy across various domains, including the General Social Survey, Big Five personality traits, economic behavioral games, and experimental replications. The study also investigates and mitigates potential biases in agent accuracy across demographic groups.

  1. Key Points

  • Developed a novel agent architecture that simulates the attitudes and behaviors of 1,052 real individuals.

  • Used large language models and two-hour qualitative interviews to create generative agents.

  • Achieved 85% accuracy in replicating participants’ responses on the General Social Survey (GSS), comparable to participants’ own replication rate.

  • Agents performed comparably to participants in predicting personality traits and outcomes in experimental replications.

  • The architecture significantly reduced accuracy biases across racial and ideological groups compared to agents based solely on demographic descriptions.

  • Created a two-pronged access system for the resulting agent bank: open access to aggregated responses and restricted access to individual responses.

  • Interviews significantly improved agent predictive accuracy compared to using demographic or persona-based descriptions.

  1. Notable Quotes

(None explicitly identified in the provided PDF excerpt)

  1. Primary Themes

  • Human Behavioral Simulation: The core theme is the development and evaluation of a novel method for simulating human behavior at scale.

  • Generative AI in Social Science: The study explores the potential of generative AI to advance research in social sciences by enabling large-scale simulations and controlled experiments.

  • Bias Mitigation in AI: The research directly addresses concerns about bias in AI models, demonstrating methods for reducing such biases in the context of human behavior simulation.

  • Data Access and Replicability: The creation of a publicly accessible (with appropriate safeguards) agent bank highlights the importance of data sharing and reproducibility in scientific research.