Music and Sound
Music is incredibly powerful. It can help us focus, lift our mood and even change the way we move. Previous works have explored how music changes the way runners run, synchronise their movements and even dance. In our work we have been exploring how incorporating music into real-time feedback of our movement can alter both the way we move and our perception of our own movement, to help people who struggle with engaging in regular physical activity.
Sound feedback for movement, also known as movement sonification has been used in the past to support athletes, people learning new movements and in rehabilitation. However, it is primarily focused on informing people about their movement, for example how far have you gone. But we know that while information is important for people struggling with exercise, they may also need extra encouragement to feel a sense of accomplishment.
That is where the music comes in! We have been looking at an idea in music called musical expectation, which is how, based on our previous experience and music we are hearing, we expect the music to continue. This is often sensed strongly at the end of a piece of music where a certain kind of cadence (a musical ending) may make us feel like the song has come to an end, or that we should expect it to continue on.
How does music change how we perceive feedback?
We have developed the Movement Sonification Expectation Model (or MoSEM) to help us explore how those feelings of musical tension or resolution may alter our movement perception when they come from musical feedback generated by our own movement. By having the music be part of the feedback, we are no longer simply listening to the music we become an active agent in creating it. The leads to two main instances of the MoSEM when it comes to music feedback:
1) the music resolves at the end of the movement and we feel a sense of completeness and reward;
2) the music is incomplete and hence we feel encouraged to continue our movement.
Our studies of how this model applies to people’s perception of their movement confirm these ideas, but also show us something else. We found that when there were strong visual cues in the environment, these impacted how much impact the musical feedback had on people’s movement. This may be attributed to people’s “visually dominant” nature, in that we typically believe what we see more than what we hear. Nonetheless, this work demonstrates how musical feedback may be a useful tool to support people’s physical activity, which could be people trying to get into an exercise routine or for people doing physical rehabilitation exercises. The impact of external cues on how musical feedback can change our perception of our own movement.
You can find the full paper, “Movement sonification expectancy model: leveraging musical expectancy theory to create movement-altering sonifications” here: http://link.springer.com/article/10.1007/s12193-020-00322-2
Joseph W. Newbold 