When AI thinks about culture: comparing language-based and empirical measures of individualism–collectivism

Abstract Large language models (LLMs) are increasingly used to generate cultural insights, yet little is known about their cultural expertise and the extent to which their outputs replicate or correct for cultural stereotypes. This study compares country-level estimates of individualism–collectivism (I-C) derived from five different LLMs with a recently validated I-C index based on nationally representative surveys. The LLM-based estimates correlate highly with the survey-based index (up to r = 0.874) but also exhibit systematic deviation from the benchmark: Western societies are consistently overestimated on individualism, while Confucian East Asian societies are underestimated. Statistically, the type of benchmark-relative deviations suggests group-specific baseline distortions rather than random noise, with smaller and older models showing weaker alignment. Moreover, the deviations are systematically aligned with Hofstede’s original I-C rankings, suggesting that LLM-generated cultural representations may reflect information from legacy cultural frameworks alongside contemporary empirical patterns. These findings demonstrate both the potential and the risks of using LLMs as cultural interpreters. In consequential settings, LLM-generated cultural insights should be used responsibly—with calibration, human oversight, and awareness of potential biases. Used uncritically, they risk reinforcing outdated narratives that misdirect academic research and managerial decisions.