“In this project we find one great beauty of AI intermingling with art: the emergence of a toolkit for collaborating with a model of cultural history—one that is accessible and allows for experimentation and rapid evolution in the present.”
—Sarah Schwettmann, Project Co-Lead and PhD Student, MIT Brain and Cognitive Sciences
In a collaboration between New York’s Metropolitan Museum of Art (The Met), Microsoft, and MIT, researchers implemented machine learning models of the structure underlying different categories of artworks in the museum’s digital collection. The team then built an interactive web studio where visitors could explore and experiment with that structure.
Gen Studio places images of existing Met artworks onto a digital map, then uses trained neural networks known as generative adversarial networks (or GANs) to combine features. As visitors explore the map, they see new images generated from combining The Met’s images, weighted by the distance each one is from the other on the map. This creates an experience designed to simultaneously convey both the uniqueness of each image and its potential to be iterated. As a final step, visitors can search for works in The Met collection that are visually similar to a generated image.
In February 2019, the team presented this project in a showcase in the Great Hall of The Met Fifth Avenue in New York City. Visitors were able to step into the latent space of The Met collection via an immersive, projected map of existing images, their nearest-neighbors in latent space, and sequences of images interpolating between them.
“In the sets of interstitial images, we find a dreamlike complement to The Met collection: its features manifest in new forms, with combinations both strange and familiar,” says project co-lead Sarah Schwettmann. “In the models, we find potential energy; the suggestion, just beneath the surface, that an infinite number of such images could be created, then taken as inspiration to refine the models themselves—a feedback loop befitting a 21st century articulation of iterative creativity.”
Project leads:
Sarah Schwettmann, Project Co-Lead and PhD Student, Department of Brain and Cognitive Sciences, MIT
Matthew Ritchie, 2018-20 Dasha Zhukova Distinguished Visiting Artist at the Center for Art, Science & Technology, MIT
Collaborators:
Maddie Cusimano, PhD Student, Department of Brain and Cognitive Sciences, MIT
Luke Hewitt, PhD Student, Department of MIT Brain and Cognitive Sciences, MIT
Samuel Klein, Knowledge Futures Group
MIT undergraduate externs also worked on this project:
Gillian Belton ’19, Department of Electrical Engineering and Computer Science (EECS), MIT
Darius Bopp ’20, EECS, MIT
Diana Nguyen ’20, EECS, MIT
Alonso Salas Infante ’21, EECS, MIT
Elaine Zhang ’20, EECS, MIT
Launched: February 2019
On view: Experience the project online
More information
How Artificial Intelligence Can Change the Way We Explore Visual Collections
MIT News: New collaboration sparks global connections to art through artificial intelligence