Where AI Meets Hip-Hop
Colby Assistant Professor Ben Baker studies the data of dance
Jump, land, step, repeat.
A few lines of the song “Mask Off” by Future play again and again as two dancers move to the beat. This is no ordinary dance practice, though. Cameras in the production room at the Gordon Center for Creative and Performing Arts record the dancers’ every move. Across the room, Assistant Professor of Philosophy Ben Baker hovers over a computer.
Images from the cameras will become 3D models of the dancers’ bodies, with important features such as the joints and limbs identified. Artificial intelligence will analyze these 3D models to learn about aspects of the dancers’ movements.
Baker said there are “two main things I feel powered to look at right now.” One is the differences between genres, such as house or breakdance. And another is what happens as someone learns a new move, a complex sequence of moves, or as someone practices a style of movement over months. “What’s the signal of someone getting better?” wondered Baker, who is affiliated with Colby’s Davis Institute for Artificial Intelligence.
He plans to study dancers as they learn one new move.
He doesn’t hide behind his computer, though. He’s a dancer himself. Hannah Junn ’25 was one of the two students helping out with his research that day. As they danced, “[we] could see him sitting there, moving his arms,” she said. So they taught him the choreography. He got it in “maybe five minutes,” said Junn. Then all three of them danced together for the cameras.
Remixing philosophy, science, and art
“I’ve been dancing since I was a kid,” Junn said. “I have Korean cultural background. … I grew up with K-pop.” A big part of that world, she said, is dance. And K-pop dance draws heavily from hip-hop.
When she got to Colby, no clubs or teams were performing these types of dance. So she and a friend started a group called Cozy at Colby. They meet in the Harold Alfond Athletics and Recreation Center once a week and teach hip-hop moves and choreography, for fitness as well as for fun. “We have a lot of regulars, and we have new people who try it out with friends,” said Junn.
When professor Baker asked Junn’s group to help with his research, she was excited but also surprised. To her, it didn’t seem that philosophy, AI, and dance had much in common. “I was wondering how this would all mesh into one,” she said.
It took Baker himself several years to realize that he could combine all these different interests. He started out studying philosophy and cognitive science, which is the study of how minds work. While he was getting his Ph.D. in philosophy at the University of Pennsylvania, he danced with the Academy of Phresh in Philadelphia. Once, they performed in the home ballpark of the Philadelphia Phillies baseball team. “It was a cool moment to be on that stage,” he recalled.
A few years into his Ph.D., he realized that he could study how the mind works through the lens of dance. “It sort of clicked,” Baker said. So he began focusing on embodied cognition. That’s “the kind of intelligence manifested when we dance, when we do athletic things, when we navigate space,” said Baker. These investigations led him to machine learning. He realized he could use AI technology as a tool to explore the questions he was most interested in.
To Baker, AI “is a powerful statistical tool.” He doesn’t shy away from calling it intelligent, but he said it’s a very different type of intelligence than what people possess. When people make AI out to be more human than it is, he said, this “lessens the image we have of ourselves.” People have unique abilities that they “manifest in amazing ways every day,” said Baker. When we’re empowered by this understanding, we can use AI to accomplish incredible things, he said
Teaching AI hip-hop styles
Baker joined Colby’s faculty in 2023. Beste Kuruefe ’26 met him soon after he arrived. She’s a computer science major with a concentration in AI. “I really like patterns. I actually like rules,” she said.
And like Baker, she’s also a dancer. In high school in Istanbul, Turkey, she saw a modern dance show and was hooked. “There’s something about movement that I really like,” she said. “But at first I was seeing it as more of a hobby.” When she started learning about Baker’s work, though, she realized it was possible to use AI to study dance. “I was like, yay! This is so cool,” she said.
Now, Kuruefe works as Baker’s research assistant. In 2024 she helped him with a new study. “We were trying to use machine learning models to be able to classify dance movements into their genres,” she explained. For this study, Baker didn’t film anyone dancing. He used an existing dataset that includes 1,408 3D models of dance movements from 10 different hip-hop styles. He used this data to teach a machine learning model to tell the difference between the styles.
In machine learning, an AI model has to go through training. This is when the model looks at example data and learns to recognize important features. Usually, AI developers let a model discover features on its own. If he had done this, Baker said, he’d have an AI model that could identify dance genres. But “you can’t learn as much from it,” he said. The features an AI model comes up with on its own tend to be extremely difficult for people to understand. It may be impossible to explain why the model came up with its output. People often call this type of AI a “black box.”
Baker didn’t want a black box. He wanted something he could understand. So, using his knowledge of dance, Baker defined a set of 17 features for the AI model to look for in the data. These included things like how the ankles or wrists moved, or how the body twisted. Once the model had learned to recognize dance styles using these features, Baker could ask interesting questions, such as how important each feature was to each style. The most important feature for all the styles was expandability. This is a measure of how far all the joints are from the sacrum at the base of the spine. “I thought it might carry a lot of information,” he said. And the study proved him right.
Other results were more surprising. He created a 3D map that plotted the different dance styles in a space defined by their most important features. So, similar genres were located near each other. Interestingly, Baker found “a big empty space in the center of the plot.” This could be an existing genre that wasn’t in the dataset. Or maybe it’s a type of motion that “doesn’t look good or doesn’t feel good to execute,” Baker said. Or it could be a genre that does not currently exist but might someday.
Next steps
Now, with the help of Cozy at Colby, Baker is taking this research even further. By capturing his own data, he’ll be able to ask even more interesting questions about dance.
He plans to record a dancer as they learn and perfect a new move, a sequence of moves, or a new style of movement over time. Then, he can look at how the features change as a dancer is learning. This could lead to new tools for dance education or training, he said.
He also plans to get his AI model working in real time. That way, he’ll be able to see what features the AI detects as a dancer moves. Kuruefe is planning to help with this project as well. “I’m excited to get into that,” she said.
Real-time AI analysis could help reveal what that empty space on the map is all about. Also, choreographers could use the tool to develop new sequences of moves that intentionally explore parts of the feature space. “I feel like dancers, choreographers are often like, okay, what ruts am I stuck in?” said Baker. Perhaps an AI tool like this one could help get them unstuck. It could help inspire creative new ideas in dance.
Junn appreciates that approach. She said, “It’s just really interesting that somebody could have the idea to do this sort of research.”
