Neoperceptions

Augmenting human perception with live performance technology

Neoperceptions brings together art and science to create new live performance technology that allows participants to “hear color” and “see sound.”

Composer Mary Bichner has polymodal synesthesia: she associates musical notes with specific colors. The Neoperceptions project creates a means for people without synesthesia to experience the color palette that Bichner sees, and takes inspiration from, when composing music. During a live performance, visual projections and costumes woven with LEDs are activated exactly in time with Bichner’s music, allowing the viewer to experience an audiovisual environment in which pitch and color are intertwined.

This new technology was presented in a live performance, Synesthesia Suite, at the Hayden Planetarium at the Museum of Science in April 2019 and supported in part by a CAMIT grant. Musicians Bichner (piano, vocals), Sonia Deng (violin), Anna Stromer (viola), Dominick Douglas (viola), and Timothy Paek (cello) performed works by Bichner, Mozart, and others. The performers wore audio-responsive wearables developed by the Neoperceptions team and were accompanied by room illuminators created by Arizona State University SciHub, and 360° videos created by the Museum of Science team for the Hayden Planetarium dome.

More at the project website: Neoperceptions

Neoperceptions brings together physicists, musicians, engineers, and designers to create innovative technology for live performance.

Frank Wilczek, Herman Feshbach Professor of Physics, MIT and Professor of Physics, Arizona State University; 2004 Nobel Laureate

Siranush Babakhanova, MIT ‘20 (Physics, Computer Science), Synthetic Neurobiology Group, MIT Media Lab

Thomas Sanchez Lengeling, Media Arts and Sciences SM ‘17; Research Affiliate, City Science Group, MIT Media Lab

Brodi Elwood, Axion researcher, MIT Kavli Institute for Astrophysics and Space Research; MIT ‘19 (Physics)

Mary Bichner, award-winning orchestral composer with synesthesia

Nathan Newman, Professor of Solid State Sciences, Materials Program, Arizona State University

Brent Smith, Graduate Research Associate, School of Earth and Space Exploration, Arizona State University SciHub

Terry Kang, MIT ‘22

Justin Pye, Research Professional, Arizona State University SciHub

Alexandra Rieger, PhD student, Opera of the Future Group, MIT Media Lab

Stephanie Brandt, PhD student, Physics, MIT

Anne Dominic, Office Specialist, Arizona State University

More at the artists’ website: Neoperceptions

Past Events

Synesthesia Suite
Thursday, April 4, 2019
Museum of Science
1 Science Park, Boston, MA

Neoperceptions at MIT Design Week
Friday, April 26, 2019
Long Lounge, MIT Building 7, Room 429
77 Massachusetts Avenue, Cambridge, MA
Presented as part of the MIT Design Week 2019 Showcase

REVEAL Challenge Expo
Friday, June 21, 2019 / 10–11:30 AM
MIT List Visual Arts Center
20 Ames Street, Cambridge, MA
Presented as part of MIT Museum’s collaboration with Nord Anglia Education

Siranush Babakhanova and the garment she designed, with electronics by Thomas Sanchez Lengeling and Terry Kang.