Machine Meets Man: Evaluating the psychological reality of corpus-based probabilistic models

Divjak, Dagmar, Dabrowska, Ewa and Arppe, Antti (2016) Machine Meets Man: Evaluating the psychological reality of corpus-based probabilistic models. Cognitive Linguistics, 27 (1). pp. 1-33. ISSN 0936-5907

[Cognitive Linguistics] Machine Meets Man Evaluating the Psychological Reality of Corpus-based Probabilistic Models.pdf - Published Version

Download (6MB) | Preview
Official URL:


Linguistic convention typically allows speakers several options. Evidence is accumulating that the various options are preferred in different contexts, yet the criteria governing the selection of the appropriate form are often far from obvious. Most researchers who attempt to discover the factors determining a preference rely on the linguistic analysis and statistical modeling of data extracted from large corpora. In this paper, we address the question of how to evaluate such models and explicitly compare the performance of a statistical model derived from a corpus with that of native speakers in selecting one of six Russian TRY verbs. Building on earlier work we trained a polytomous logistic regression model to predict verb choice given the sentential context. We compare the predictions the model makes for 60 unseen sentences to the choices adult native speakers make in those same sentences. We then look in more detail at the interplay of the contextual properties and model computationally how individual differences in assessing the importance of contextual properties may impact the linguistic knowledge of native speakers. Finally, we compare the probability the model assigns to encountering each of the six verbs in the 60 test sentences to the acceptability ratings the adult native speakers give to those sentences. We discuss the implications of our findings for both usage-based theory and empirical linguistic methodology.

Item Type: Article
Subjects: Q100 Linguistics
Department: Faculties > Arts, Design and Social Sciences > Social Sciences
Depositing User: Ay Okpokam
Date Deposited: 02 Mar 2016 12:41
Last Modified: 01 Aug 2021 07:38

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics