Insights into the accuracy of social scientists’ forecasts of societal change

Grossmann, Igor, Rotella, Amanda, Hutcherson, Cendri A., Sharpinskyi, Konstantyn, Varnum, Michael E.W., Achter, Sebastian, Dhami, Mandeep K., Guo, Xinqi Evie, Kara-Yakoubian, Mane, Mande, David R., Louis, Raes, Tay, Louis, Vie, Aymeric, Wagner, Lisa, Adamkovic, Matus, Arami, Arash, Arriaga, Patricia, Bandara, Kasun, Banik, Gabriel, Bartoš, František, Baskin, Ernest, Bergmeir, Christoph, Bialek, Michal, Børsting, Caroline K. and Browne, Dillon T. (2023) Insights into the accuracy of social scientists’ forecasts of societal change. Nature Human Behaviour, 7 (4). pp. 484-501. ISSN 2397-3374

Grossmann_et_al.preprint.pdf - Accepted Version

Download (4MB) | Preview
Official URL:


How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. Following provision of historical trend data on the domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N=86 teams/359 forecasts), with an opportunity to update forecasts based on new data six months later (Tournament 2; N=120 teams/546 forecasts). Benchmarking forecasting accuracy revealed that social scientists? forecasts were on average no more accurate than simple statistical models (historical means, random walk, or linear regressions) or the aggregate forecasts of a sample from the general public (N=802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models, and based predictions on prior data.

Item Type: Article
Additional Information: Funding information: This program of research was supported by Basic Research Program at the National Research University Higher School of Economics (M. Fabrykant), John Templeton Foundation grant 62260 (I.G. and P.T.), Kega 079UK-4/2021 (P.K.), National Center for Complementary & Integrative Health of the National Institutes of Health under Award Number K23AT010879 (Simon B. Goldberg), National Science Foundation RAPID Grant 2026854 (M.E.W.V.), PID2019-111512RB-I00 (M.S.), NPO Systemic Risk Institute (LX22NPO5101) (I.R.), Slovak Research and Development Agency under contract no. APVV-20-0319 (M.A.), Social Sciences and Humanities Research Council of Canada Insight Grant 435-2014-0685 (I.G.), Social Sciences and Humanities Research Council of Canada Connection Grant 611-2020-0190 (I.G.), Swiss National Science Foundation grant PP00P1₁₇₀₄₆₃ (O. Strijbis).
Uncontrolled Keywords: meta-science, forecasting, expert judgement, political polarization, prejudice, well-being
Subjects: L300 Sociology
Department: Faculties > Health and Life Sciences > Psychology
Depositing User: John Coen
Date Deposited: 07 Feb 2023 08:55
Last Modified: 09 Aug 2023 03:30

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics