How musical feedback can change the way we move

printed musical note page

Music and Sound

Music is incredibly powerful. It can help us focus, lift our mood and even change the way we move. Previous works have explored how music changes the way runners run, synchronise their movements and even dance. In our work we have been exploring how incorporating music into real-time feedback of our movement can alter both the way we move and our perception of our own movement, to help people who struggle with engaging in regular physical activity.

Sound feedback for movement, also known as movement sonification has been used in the past to support athletes, people learning new movements and in rehabilitation. However, it is primarily focused on informing people about their movement, for example how far have you gone. But we know that while information is important for people struggling with exercise, they may also need extra encouragement to feel a sense of accomplishment.

That is where the music comes in! We have been looking at an idea in music called musical expectation, which is how, based on our previous experience and music we are hearing,  we expect the music to continue. This is often sensed strongly at the end of a piece of music where a certain kind of cadence (a musical ending) may make us feel like the song has come to an end, or that we should expect it to continue on.

How does music change how we perceive feedback?

We have developed the Movement Sonification Expectation Model (or MoSEM) to help us explore how those feelings of musical tension or resolution may alter our movement perception when they come from musical feedback generated by our own movement. By having the music be part of the feedback, we are no longer simply listening to the music we become an active agent in creating it. The leads to two main instances of the MoSEM when it comes to music feedback:

1) the music resolves at the end of the movement and we feel a sense of completeness and reward;

2) the music is incomplete and hence we feel encouraged to continue our movement.

Our studies of how this model applies to people’s perception of their movement confirm these ideas, but also show us something else. We found that when there were strong visual cues in the environment, these impacted how much impact the musical feedback had on people’s movement. This may be attributed to people’s “visually dominant” nature, in that we typically believe what we see more than what we hear. Nonetheless,  this work demonstrates how musical feedback may be a useful tool to support people’s physical activity, which could be people trying to get into an exercise routine or for people doing physical rehabilitation exercises. The impact of external cues on how musical feedback can change our perception of our own movement.

You can find the full paper, “Movement sonification expectancy model: leveraging musical expectancy theory to create movement-altering sonifications” here: http://link.springer.com/article/10.1007/s12193-020-00322-2

Submission, GetaMoveOn and CHI2019

A Busy Month for me!

At the start of April, I submitted my PhD Manuscript “Musical expectancy within Movement Sonification to Overcome Low Self-efficacy”. It is currently under examination and I should have updates soon :D.

Secondly, I have started work with Anna Cox as a research fellow, exploring musical displays to reduce sedentary behaviour in desktop workers.  I also joined the GetaMoveOn Fellowship program, with our first event happening this month!

Finally, I just returned from CHI 2019, where I took part in a great speech interaction workshop, led by Leigh Clark, saw Ana Tajadura-Jiménez present our paper As Light as You Aspire to Be: Changing Body Perception with Sound to Support Physical Activity” and had great discussions with old friends and new.

Now it is time to look forward to the months to come, where a lot of new projects are starting and it is an exciting time for Audio and HCI!

Audio Interaction Design

Hi!

I am a PhD candidate and researcher interested in how people interact with technology and how the way we design technology, specifically sound, can change the way in which we interact with it.

I have done this in my PhD research explore how musical feedback may be used to alter physical activity: My PhD research

And in many other projects explore how we might use audio to design richer and supportive interactions: My Projects

post