Surprise Machines
More than 200,000 objects from the Harvard Art Museums, sorted by visual similarity into a single nebula on the nine-monitor wall of the museum's Lightbox Gallery.
Surprise Machines set out to show the image collection of the Harvard Art Museums all at once: about 213,000 digitized objects, of which less than one percent have ever been on view in the galleries. The name comes from Alan Turing’s reply to Ada Lovelace — machines, he argued, take us by surprise with great frequency.
The map is built with PixPlot from the Yale Digital Humanities Lab: each image passes through an Inception neural network, and UMAP places visually similar objects side by side, so the collection spreads out as a nebula of clusters — Bauhaus drawings, powdered pigments, a patch of hand-made portraits in which Gandhi and Einstein turn up.
For the nine-monitor wall of the museum’s Lightbox Gallery the team switched to Trails, a lighter DHLab viewer; a gesture-driven “choreographic interface” was developed in parallel, but the show ran with an air mouse.
It was shown in May 2022 as the closing week of Curatorial A(i)gents, metaLAB (at) Harvard’s series of machine-learning experiments in the Lightbox Gallery, and written up in the Information Design Journal. I joined as a collaborator, helping build the prototype and bringing my experience with interfaces for large image collections.
Frontal photos of the wall (nebula and Trails views): Sarah Newman / Information Design Journal, CC BY 4.0.
