Control of footstep sounds using real-time motion caption

This master thesis project is offered by the Sound in Interaction group at KTH [1][2].

BACKGROUND
The work of this master thesis is in the framework of two projects: the SOM project [3] funded by VR - The Swedish Research Council, and the Ljudparken project [4], funded by PTS - The Swedish Post and Telecom Authority.
The thesis work is part of growing research fields known as data sonification, embodied music cognition and mediation technology. We are observing an increasing need for the development of theories, models, and tools for representing human movements by means of sound, as a complement to visualization techniques.
By using development tools which are at the state-of-the- art in the field of Sound and Music Computing, it is possible to (1) Develop models for communication of human movement information by means of sound; (2) Investigate how sound feedback can influence motor learning; (3) Realize real-time audio feedback to human movements. Starting point are theories and methods from the Interactive Sonification research field. The final goal is the realization of new models for interactive sonification of human movements for understanding, modeling, and improving human interaction with sound.

GOAL
You will design, implement and test a new software tool for the real-time control of footstep sounds. The sounds will reflect the characteristics and actions of a human walker being tracked using a motion caption system such as Natural Point OptiTrack [5], or Microsoft Kinect [6]. The sounds will simulate different materials of both shoes and floor. For generating the sounds you will start from the Sound Design Toolkit [7].

Potential applications of the new tool are in therapy, computer games, and sound effect for film and theater.

The work will be in collaboration with Radioteatern [8] at the Sveriges Radio.

LEARNING OUTCOMES
After fulfilling the master thesis, you should be able to:
1. determine prerequisites for and plan a larger project,
2. apply the relevant knowledge and skills, which are acquired within the technical area, to a given problem,
3. determine if the project is viable and collect the knowledge to run it,
4. pursue the project according to plan and continuously handle and document deviations from the plan in a professional way,
5. make oral and written presentations of the project in a form that meets established norms with regard to structure, language and content,
6. reflect on, evaluate, and critically assess one’s own and others’ scientific results,
7. show increased knowledge in the areas of Sound in Interaction, Sound Design, Sound and Music Computing [9]
8. evaluate the functioning of sound-based interactive applications
9. use state-of-the-art software tools and sensors for the design of new sound-based interactive applications

LINKS
[1] http://www.kth.se/student/kurser/kurs/DT212X?l=en_UK
[2] http://www.speech.kth.se/music/sound_projects.html
[3] http://www.speech.kth.se/projects/project_details.html?ID=185&user_tag=proj_185
[4] http://www.speech.kth.se/projects/project_details.html?ID=190&user_tag=proj_190
[5] http://www.naturalpoint.com/optitrack/
[6] http://en.wikipedia.org/wiki/Kinect
[7] http://www.soundobject.org/SDT/
[8] http://sverigesradio.se/sida/default.aspx?programid=965
[9] http:// www.smcnetwork.org

keywords: music@kth, tmh@kth, [email protected], sound in interaction, sound design