Research

Since 1993, I have been conducting research in novel interfaces for multimedia performance, musical expression, wearable sensing, and Affective Computing. While a graduate student at the MIT Media Lab, I developed the Digital Baton and Conductor's Jacket sensor interfaces, which have been used both for research and performance applications. Here are some of my active research projects:

Conducting Robots -- an innovative course funded by the National Science Foundation that emphasizes teamwork and creativity in the development of nonhuman systems to conduct the TCNJ Orchestra.

Computer Vision Analysis of Conducting Gestures -- ongoing collaborations with Dr. Andrea Salgian of the TCNJ Computer Science Department to extract meaningful information from video recordings of conductors.

Virtual Maestro -- an interactive conducting system that uses the Wii-remote as a conducting baton. Version 2.0 premiering at Ravinia Music Festival near Chicago in Summer, 2014. (Press Reviews in The UK Guardian, etc.)

Boston Symphony Orchestra collaboration -- In April 2006, I teamed up with researchers at McGill University and Immersion Music, Inc. to deploy a large-scale technical experiment in a live concert of the Boston Symphony Orchestra. Working from an hypothesis that emotion, in the form of affective energy, is communicated and contagiously conveyed through the medium of a musical performance, the experiment was designed to reveal some of the mechanisms of communication between conductor, musicians, and audience. Sensors for this project were donated by Delsys Inc. and BodyMedia, Inc. During the concert, many sensors (EMG, EKG, GSR, accelerometers) were worn by conductor Keith Lockhart in an updated Conductor's Jacket. Five orchestra musicians and fifteen members of the audience also wore sensors. In addition, 30 members of the audience manipulated slider boxes registering their emotional experience. A month later, a high-resolution video of the concert was played at McGill University and another set of audience data was recorded. This project resulted in several research publications, an was featured in several television newscasts and an episode of State of the Arts by NJN (New Jersey Public Television):

The Conductor's Jacket -- Since 1997 Dr. Nakra has been investigating and using physiological sensors for research and performance applications. Numerous versions of the jacket have been used and worn by conductors in a variety of rehearsals and performances. The jacket has been presented in concerts of the Boston Pops Orchestra, Boston Modern Orchestra Project, American Composers' Orchestra, and BBC Scottish Symphony. It has been featured on CNN Headline News, the BBC World Service, the New York Times, International Herald Tribune, and other news media. In 2010 the Conductor's Jacket was nominated and voted on by Alumni of MIT to be included in the MIT150 Exhibition, a year-long celebration of MIT's founding. It is on display in the Compton Gallery at MIT through 2011. For more information, see:

1998 MIT News Office Concert announcement

1998 longer description of concert and jacket

Teresa Nakra's 2000 doctoral dissertation on the Conductor's Jacket

You're the Conductor -- an interactive installation, built in 2003 for the Boston Children's Museum in collaboration with Professor Jan Borchers of RWTH Aachen, this exhibit allows museum goers to "conduct" a video of the Boston Pops Orchestra with an infrared baton (sponsored by National Instruments and Hamamatsu). For more information, see:

youretheconductor.com

Symkus, E. Children's Museum provides a music intro: New exhibit gives kids the chance to conduct the Boston Pops. Town Online (Herald Media Boston), June 25, 2003.

Edgers, G. Children's Museum, BSO create $2 million music exhibit. The Boston Globe, June 27, 2003.

Feinberg, J.. In Tune: Boston Symphony reaches out to children. The Patriot Ledger, June 28, 2003.

Deahl, A. Striking a chord with kids. The Boston Globe, June 2003.

Brain Opera -- Tod Machover's interactive, musical journey into the mind, presented simultaneously in physical venues and online. The production premiered at Lincoln Center and toured Europe, Asia, the United States and South America from 1996 to 1998. I contributed to the project as a member of the development team and performer. I can be seen performing with the Digital Baton in this video produced by the Ars Electronica Festival (beginning at 0:44"):