Music and dance are deeply interconnected. While dance has traditionally followed the patterns, tempo, and style of music, the reverse relationship (i.e. using movement to generate or influence music) remains underexplored. Be the Beat is a boombox augmented with a camera and artificial intelligence that dynamically recommends music based on a dancer’s movements.
Drawing inspiration from hip-hop dance circles and a DJ–dancer interplay, Be the Beat extracts dance features from video, such as movement velocity, acceleration, range of motion, joint angles, and BPM. These features are then translated by a large language model into dance style descriptors which are used to either generate new music or query a music catalog for tracks that align with the dancer’s style, energy, and tempo. This generative experience between music and motion enables dancers to engage in an ongoing creative dialogue with the machine. Without needing to rely on hardcoded mappings between gestures and music synthesis, Be the Beat leverages the expressive capabilities of large language models to interpret a broad, open-ended range of gestures and body movements, vastly expanding input possibilities.
In Be the Beat, new interactions can happen at various scales. On one end, dancers can have control over the type of rhythm, fluidity, and energy of the song that is being suggested, and continuously fine-tune their movements to match what they have in mind. On the other, dancers can more easily explore new genres and choreographies, rather than strictly adhering to the constraints of a specific song. This creates a dynamic and expressive feedback loop where movement informs music, and music, in turn, inspires further movement.
