Gesture Follower
What is the gesture follower ?
The gesture follower is a ensemble of Max/MSP modules to perform gesture recognition and following in real-time.
It is integrated in the toolbox MnM of the library FTM.
The general idea behind it is to get parameters from the comparison between a performance and an set of prerecorded examples.
The gesture follower can guess the two following questions:
- which gesture is it ? (if you don't like black and white answers, you can get "greyscale" answers: how close are you from the recorded gestures ? )
- where are we ? (beginning, middle or end of the gesture)
What is a gesture anyway ?
A gesture here can be any multi-dimensional temporal curve, sampled at relatively low frequency compared to sound. With the current implementation in Max/MSP the frequency sampling period must be at least 1 milisecond, but typically, 10-20 milisecond is recommended. There are no upper limit (if you have time...).
There is no technical limit for the dimension of the gesture data (or number of sensor channel) other than what your computer can afford as a CPU load (for example 20 is generally no problem).
In Max/MSP the data feeding the gesture follower can be taken from a list, for example:
- sound parameters (pitch, amplitude, etc)
- mouse, joystick coordinates
- parameters from video tracking (EyesWeb, Jitter, etc)
- Wiimote
- MIDI
- any sensors data, etc...
- any combination of the above (you said multimodal ?)
Credits
Gesture Follower is a project developed by Ircam IMTR team.
See official web page for details.