Volumetric Cinema Explores the 360-Degree 3D Holographic Experience

Posted on May 8, 2019 by Arts at MIT

Michael Bove and David Levine collaborate with MIT students to push the boundaries of 3D cinema.

Imagine walking into a darkened room. As your eyes adjust, a figure, floating in space, beckons. You get closer. The story begins.

Welcome to Volumetric Cinema, and the entrancing visualization of 360-degree 3D holographic cinema.

The project began in a January 2019 IAP class led by V. Michael Bove, Principal Research Scientist and head of the Object-Based Media Group at the MIT Media Lab, and David Levine, Professor of the Practice of Performance, Theater, and Media at Harvard University. The research-driven exploration continues through 2019, culminating with the debut of a film produced on the Voxon VX1, a cutting-edge 3D volumetric display.

Bove and Levine are co-recipients of an MIT Center for Art, Science & Technology 2018 Mellon Faculty Grant. Working in close collaboration with students from MIT, Harvard, and Wellesley, they’re testing the limits of a new technology that pays homage to pre-digital cinema.   

We spoke with Levine and Bove about Volumetric Cinema in the MIT Media Lab.

Bove: This spring we’re researching just how far we can augment the Voxon’s capabilities.

Levine: We’re really pushing the technology. And then we do post-production in the fall and debut the film in January 2020.

Bove: I’m expecting the film will be significant in length—perhaps 20 minutes. It’s not going to be a quick little demo. Because we want people to spend some time getting into the space and all of the different visual qualities. We have to educate the viewer on how to see this film. It’s like 1905, when you’d go to a theater and see your first projected film on the screen. You spend the first few minutes trying to understand how they even tell stories on it. The challenge is much the same with this new technology—the story has to bring people into the visual language.

Levine: This thing is about nothing but its limitations. And you’re making an aesthetic out of what you have. When I first looked at the Voxon, they kept apologizing for these imperfections. And I was like, no, they’re amazing. What really intrigued me was the technology’s weirdly retro, throwback qualities.

Bove: It’s not perfect. It’s not perfect in some really charming ways.

Levine: Exactly.

Bove: It turns out that volumetric displays have been around for more than 100 years. And for a variety of reasons, they haven’t been used in this way. Today’s technology is a whole lot cheaper than anything that was previously available. You could have one of these units for a few thousand dollars. In fact, we’re exploring another approach that’s a lot cheaper, which we’ll display this spring.

When we started the project we weren’t sure if we were going to use a game engine, which makes it fairly easy to generate dynamic 3D content with various kinds of responsiveness. The Voxon didn’t have a way of running a game engine on their hardware. Now we’re almost certain we can do that. Mapping that onto a volumetric display makes perfect sense.

Levine: I mean, it’s so intensely mechanical, you know? It’s not screen-based. It makes a sound like an old film projector, which I find extremely intriguing.

Bove: And the imagery is a little bit grainy in a film kind of way.

Levine: Once you start working with volume, the pixels with volume are essentially grainy. In contrast, CGI can be so perfect. But with this technology, the form doesn’t vanish in favor of content. In fact, the form itself is at least half the content.

Bove: When you think about it, your entire storytelling space is contained in a fixed volume. With a volumetric display, there are some visual things you can do that wouldn’t make any sense on a 2D screen. You can’t have things arbitrarily far away in terms of the spatial relationship. You don’t have infinite depth.

Levine: But when you start playing with scale on this thing, it can actually begin to feel kind of infinite, partially because it’s so hypnotic, even though it’s within a limited amount of space. You generate the content, in a way, from the pathos of the form. It’s human.

Bove: To some extent, it’s more like theater in the round than filmmaking. But there are some issues with transitions. How does a character come in? How do you change scenes? How do you change viewpoint?

Levine: I tend to categorize these challenges according to freedom of movement. For me, theater and film feel exactly the same because there’s a viewing angle that’s restricted and fixed. So you’ve got this combination of three different kinds of media, between the spectator and the actual thing you’re looking at.

Bove: And then there are visual conventions that people associate with cinematic storytelling—you can do them on a volumetric display, but they just look odd or unexpected. I mean a fade doesn’t look like a fade in a film. You could do a sort of wipe, but that looks odd. We had some students play with doing a morph, where you transition to another scene kind of geometrically. At this point, we’re creating a catalog of all the things we can do with the technology.  

Levine: We’ve been working with students who are interested in exploring these inquiries as artists, programmers, animators, or software technicians. They create short animations, kind of micro-artworks, based on their elected assignments.

Bove: There was this really special thing that happened down in the IAP classroom last January. At a certain point, we turned off the lights, and people would just show their work in this darkened room. There was a really fun dynamic around the table.

Levine: Basically, we’re working like a startup company in a discovery process, which is exciting. We have questions on both the technical and design elements and about the purely cinematic language.

Bove: We’ve imagined some really crazy stuff. Like, what if the character has a sense of who is in the room and where, and they could turn toward someone and say, “Hey, you!” And then turn toward David and say, “You, too!” There are a bunch of little experiments that may or may not find their way into the final production, but I think we just have to do them to find out.

Levine: Using conventional cinematic audio that’s surround-based, which cinema-goers are accustomed to, will short circuit a little bit of what you’re expecting. But the thing this form generates is a sense of wonder. I mean, you’re walking into a dark room and experiencing this amazingly seductive technology. Hopefully, the content will live up to the seductiveness of the visuals.

 

Written by Connie Blaszczyk