A Visual Exploration of Two Museum Collections
This research project and its resulting interface builds on previous work of FW4 and VIKUS-Viewer and provides access to a selection of objects from two collections of the Berlin State Museums (SMB/SPK), comparing fine art paintings with everyday artifacts.
The overarching goal of this project is to move away from object-centeredness and reflect on the uniqueness and diversity of museum collections. A particular ambition is to reveal resonances across the boundaries of collections and to devise evocative and interactive arrangements that invite for their associative exploration.
To achieve this goal, the project uses a similarity algorithm that arranges the objects by image and title similarity in a global overview. Hand-drawn keywords, inspired by Multiplicity by Moritz Stefaner, have been added in close collaboration with collection experts to support orientation among the machine-generated arrangement. When selecting an object, the path view offers a detail view and further information while also listing other objects of the collections by descending image and title similarity. The final visualization is programmed in Svelte, D3, and PixiJS and is published as open-source on GitHub. Preliminary prototypes have been developed in ObservableHQ in order to explore the technical feasibility of the project.
The project uses Google’s BiT-M and Multilingual Universal Sentence Encoder to extract feature vectors of images and descriptions, respectively. These feature vectors are concatenated to form a large vector that represents both the imprint of the visual image and the textual description. UMAP is used to reduce the dimensionality of the vectors to two in order to plot them on an XY canvas.
Overall, the technical side of the project involves a range of techniques and tools from machine learning to data visualization and web development. By combining these different approaches, the project offers a powerful and engaging way to explore museum collections and discover connections between objects that may not be apparent in traditional collection interfaces.
This approach offers a more intuitive and engaging way to explore museum collections and discover connections and patterns that may not be apparent in traditional collection interfaces. By revealing resonances across the boundaries of collections, the project invites curiosity-driven browsing and offers a more complete representation of the extensive collection holdings.
Chrispie realized this project in teamwork with UCLAB Mark-Jan Bludau, Viktoria Brüggemann & Marian Dörk.