Amphibian advances the field of VR while exploring the relationship between diving and disability
“My diving bell becomes less oppressive, and my mind takes flight like a butterfly,” Jean-Dominique Bauby wrote in his agonizingly beautiful account of living with severe disabilities, The Diving Bell and the Butterfly. Holding these contrasting images of physical submersion and mental liberation in your mind is a useful way to approach Dhruv Jain’s VR project, Amphibian. Jain (MS candidate, MIT Media Lab, Living Mobile Group), who is partially deaf, created this SCUBA diving simulator to help abled people understand in an artistic way the liberating effects of disabilities, which he likens to the experience of being underwater:
“Underwater, our vital senses are dulled and warped. Visual range is limited and magnification is distorted. The senses of smell, taste and touch are severely muted. Hearing is distorted too, since sound travels five times faster in the water. But in these conditions, I experienced the kind of peace one can only expect to feel with the freedom of weightlessness. Having done 54 dives in the last 10 months, I can call myself a good diver now. And yet every time I dive, I find the underwater experience to be rejuvenating; it is emotional, and almost spiritual.”
An Experience Machine
These experiences compelled Jain to explore the connection between diving and disability. While he finds diving to be a peaceful endeavor, he realized it is too physically and mentally challenging for many people. So, he resolved to build an “experience machine” that places people in an ocean environment without their having to leave dry land. Under faculty advisor, Principal Research Scientist Chris Schmandt (Head of the MIT Media Lab’s Living Mobile Group), Jain and his team constructed Amphibian. As the name suggests, the project allows users to straddle aquatic and terrestrial worlds by providing an immersive SCUBA diving experience through a water-free simulator.
Beyond Audio and Visuals
Most existing SCUBA diving simulations in VR are limited to visual and aural displays. Amphibian advances the field of VR by engaging additional sensory modalities like thermoception (sense of temperature), equilibrioception (sense of balance), and proprioception (sense of spatial orientation and movement). Users lie on their torsos on a motion platform with outstretched arms and legs placed in a suspended harness. An Oculus Rift head-mounted display and headphones allow them to see and hear the underwater environment. Various sensors are used to simulate buoyancy, drag and temperature during the simulation. For example, Peltier modules attached to participants’ wrists through motion tracking gloves simulate temperature changes as they dive deeper into the water. An inflatable airbag placed under the torso allows the user to control ascent or descent through breathing; inhaling makes the airbag inflate and exhaling makes it deflate. The virtual body rises up and down in sync. This is done using a gas sensor attached to a snorkel that measures the amount of air inhaled or exhaled.
To navigate in the virtual world, the user wears gloves with embedded flex sensors and IMUs (Inertial Measurement Units) that track each hand’s movement. Leg motion is also tracked via IMUs and users swim forward by kicking their legs up and down. All sensor data is fed into a processing unit that converts the physical motion of hands and legs into virtual movement in the Oculus app. Another novel contribution of the system is a two-way interaction between the user and the underwater virtual world. For example, users can grasp underwater objects like rocks or crabs using their hands in a grab gesture and sense physical feedback. The sensation of picking up something is conveyed through an inflatable pad on the palm of each glove, which simulates the texture and shape of the object grabbed.
Powerful Presence
By combining available technologies in a unique way to accommodate multiple senses, the system has the potential to offer a high degree of presence in VR. (“Presence” refers to that function of immersion that increases the user’s sense of “being there.”) One of Jain’s team members, Misha Sra, reflects on how she overcame her petrifying fear of water because of Amphibian’s verisimilitude. After a traumatic experience in a pool in college, even being near water filled her with dread. She says:
“I wished I could dive but to do that, I first had to get over my fear of water and learn how to swim, which seemed like an insurmountable hurdle. Amphibian was a great middle ground.”
Freedom and Liberation
Amphibian is not Jain’s first technological system to simulate sensory deprivation nor his first artwork designed to generate empathy for disabled persons. For one of his previous projects, Blind Emporium, he created a sensory deprivation room in which outside sound and light were completely blocked, forcing participants to navigate using touch and following sounds emitted from selected objects. He found that in projects that “explore absence,” people still tended to focus on what they lacked instead of what they might gain, so he decided to alter his approach with Amphibian.
“People have a negative view of blindness or deafness, thinking that they would find it too difficult to navigate in the world, but my goal was to make people realize that disability is also liberating. When I turn off my hearing aid and close my eyes, I go into a deep meditative state,” Jain says. “Being underwater is so freeing, I thought through diving I could make people better understand the effect of disabilities.”
“Immersive Terrestrial SCUBA Diving Using Virtual Reality,” the paper explaining the technology behind Amphibian, was accepted to the “ACM Conference on Human Factors in Computing Systems (CHI),” which takes place May 7-12, 2016 in San Jose, CA. Jain received grant funding from the MIT Council for the Arts for the project, and will exhibit the work in the MIT Museum Studio, Compton Gallery.
Project team:
Dhruv Jain
Misha Sra
Jingru Guo
Rodrigo Marques
Raymond Wu
Justin Chiu
Chris Schmandt, Head of the MIT Media Lab’s Living Mobile Group