What Happens When Roboticists And Choreographers Collaborate?

Hundreds swarmed MIT’s Jack Barry Field on Sunday night to participate in “UP: The Umbrellas Project,” a joint science experiment and live art performance organized by MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) and acclaimed modern dance company Pilobolus.

The idea for the participants is simple: change the color of your LED-equipped umbrellas at the instruction of Pilobolus’ associate artistic director Matt Kent. Blue to red, red to yellow, and perhaps back again. Sometimes the choreography is specific, often times it is not.

“Get in there, greens! Okay, now everyone, surprise me!” Kent yelled from the podium on which he stood overlooking the group.

The crowd’s response to Kent’s directive is projected in real-time from footage captured by a camera hovering 65 feet overhead. The colors changed wildly, creating a kaleidoscopic fanfare of blues, greens, purples and yellows. Things got interesting once Kent ordered the group to organize themselves into some sort of pattern.

Individual participants began to take notice of the umbrellas changing color around them, in turn informing their own decisions. The performance became shaped by participants' actions and reactions to each other.

“We’re not just five, six, seven, eight-ing everything,” Kent told me in a conversation before the performance. “It’s more about understanding a vibe and being able to push a sort of atmosphere.”

In other words, it's about the group’s creative co-authorship — the very philosophy under which Pilobolus operates.

“We’ve always been interested in groups and recently we’ve become increasingly interested in how large groups function,” said Itamar Kubovy, executive director of Pilobolus. “Turns out when groups become big enough, it becomes much more effective to provide them with … algorithmic kind of instruction instead of specific instruction, as to what to do and where to go and how to behave.”

According to Kyle Gilpin, a post-doctoral associate at MIT’s Distributed Robotics Lab, a division of CSAIL, and one of the co-conceivers of UP, that is precisely why the collaboration between the lab and Pilobolus makes sense.

“Not only is it a cool art idea, but there’s actually a real research component to this,” he said. “What we like to look at is how you can control a large group of robots in a decentralized manner … The tie in to Pilobolus is that they try to have this very collaborative form of dance where there’s no real traditional choreographer that’s telling everybody exactly what to do.”

And this is where the “science experiment” comes in. While the camera projecting the crowd helps inform them of their collective creativity, it is also used to record the performance, which will then be dissected by researchers to uncover something meaningful about social dynamics and how they can be applied to artificial intelligence.

“And that’s really the research challenge," Gilpin said. "How do you take these observations of what’s going on and try to capture the essence of that and put that into 20 lines of code that tell a robot to do the same thing? So that’s kind of our jobs as researchers is to try to capture that behavior."

This year’s data will be brought back into the lab to analyze. And next year’s data will be, too. That’s because both Polobolus and MIT are working to bring the performance to different communities and countries around the world – all for the sake of learning more.