Layerage lets you create collages of r/Layer which are similar to each other. r/Layer was a community canvas for creative exploration open to everyone to create and contribute similar to r/place on Reddit. I was just so intrigued by the creativity and granularity of the layers that I had to experiment on it. Initially I built a tool to explore all those layers by zooming in and out but then switched to fiddling on this tool which could be more accessible for everyone.
You can search for a layer and hit build to create a fancy looking collage of similar layers. The size of the canvas will be automatically adjusted to you screen dimension and scaled to fit your browser window. So if you want to create a desktop or mobile background you are already set. There are some options to tweak the collage in the menu, like the canvas dimension, size and number of the layers. If you feel lucky you can hit the random button which will select a random set. On mobile you can just shake your phone to do that.
Let’s say you have all the layers which are public and you run some neural nets over them to calculate a similarity. Now look for a layer in this pool and select x amount of similar layers, distribute them to fit the canvas dimensions and voila you have a collage.
- scrape all the metadata and layers run a neural net (inception / mobilenet) on each layer to get an imprint of it
- run dimensionality reduction on all those imprints ( tSNE / UMAP) to distribute the layers by similarity in 2D
- prototype on observable notebook
- build a website using vuejs
The code is on GitHub and is open source, so you can contribute by adding a feature or suggest something in a ticket.
Running tSNE on that many vectors (around 50000x1000) can make your system swet. I was running into memory issues so I went with the WebGL version of TFJS-TSNE. Also the UMAP implementation of the YaleDHLab was helpful. Still there could be a lot finetuning to be done. Lastly, self motivation is hard. Doing the initial experiments on observable was fun and quick, but developing this app took quite some time.