Movement sonification expectancy model: leveraging musical expectancy theory to create movement-altering sonifications

Newbold, Joseph, Gold, Nicolas E. and Bianchi-Berthouze, Nadia (2020) Movement sonification expectancy model: leveraging musical expectancy theory to create movement-altering sonifications. Journal on Multimodal User Interfaces, 14 (2). pp. 153-166. ISSN 1783-7677

Newbold2020_Article_MovementSonificationExpectancy.pdf - Published Version
Available under License Creative Commons Attribution 4.0.

Download (1MB) | Preview
Official URL:


When designing movement sonifications, their effect on people’s movement must be considered. Recent work has shown how real-time sonification can be designed to alter the way people move. However, the mechanisms through which these sonifications alter people’s expectations of their movement is not well explained. This is especially important when considering musical sonifications, to which people bring their own associations and musical expectation, and which can, in turn, alter their perception of the sonification. This paper presents a Movement Expectation Sonification Model, based on theories of motor-feedback and expectation, to explore how musical sonification can impact the way people perceive their movement. Secondly, we present a study that validates the predictions of this model by exploring how harmonic stability within sonification interacts with contextual cues in the environment to impact movement behaviour and perceptions. We show how musical expectancy can be built to either reward or encourage movement, and how such an effect is mediated through the presence of additional cues. This model offers a way for sonification designers to create movement sonifications that not only inform movement but can be used to encourage progress and reward successes.

Item Type: Article
Uncontrolled Keywords: Sonification design, Musical expectancy, Movement perception
Subjects: G400 Computer Science
G500 Information Systems
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Elena Carlaw
Date Deposited: 21 Jul 2020 11:32
Last Modified: 31 Jul 2021 11:51

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics