Heterogeneity of research results: New perspectives on psychological science

Linden, Audrey H (2019) Heterogeneity of research results: New perspectives on psychological science. Doctoral thesis, Northumbria University.

Preview

Text (Doctoral thesis) linden.audrey_phd.pdf - Submitted Version Download (2MB) | Preview

Abstract

Replicability of research findings is a key issue in psychology, and has been the subject of considerable discussion by researchers. Replication is crucial to scientific principles, and underpins the consistency and verifiability of findings on which the foundation of scientific theories are built. Consistency of effects can be assessed through the perspective of heterogeneity, which indicates whether multiple research results into the same phenomenon are underpinned by the same true effect size. We use this perspective here to address concerns regarding replicability, and explore the application of heterogeneity in novel ways. This PhD project therefore aimed to: i) examine the heterogeneity of empirical findings in psychology; ii) consider the impact of biases in the research process on estimations of heterogeneity; iii) determine the strongest effects in psychology and their heterogeneity, and; iv) apply the perspective of heterogeneity to an existing psychological debate, the origin of psychological sex differences. In our first study, a re-analysis of 150 meta-analyses from cognitive, organisational, and social psychology was used to provide estimates of typical levels heterogeneity in psychology. This was compared to heterogeneity levels found across 57 multiple close replications. Our first study showed that low heterogeneity is achievable in psychology, but that typically heterogeneity estimates are high. Our next study used computer simulations to show that these levels of observed heterogeneity cannot be due to bias from questionable research practices or publication bias. In study 2, we showed the importance of including heterogeneity alongside effect size when interpreting an effect, and illustrated that even the largest mean effects in psychology might not be
consistent in direction under conditions of high heterogeneity. Finally, in study 3 we showed that heterogeneity could provide an informative new perspective to the arguments regarding the origin of psychological sex differences. We explored several possible factors that could underlie heterogeneity in research findings, and suggest that unreliable forms of measurement and low quality of metaanalyses could provide promising avenues for further investigation in this regard. Overall, this thesis has provided an overview of the application of heterogeneity to provide a new perspective on psychological research.

Actions (login required)

View Item

Downloads