Picstape - Mood-Based Image Generation for Spotify Playlists
Creative Computing MSc Final Project - 2022
Pixtape is a custom software that I developed for mood-based image generation, built upon the VQGAN-CLIP technology. The software takes Spotify playlist URLs as input and generates a text prompt for each song in the playlist, including the mood and color of the songs.
To predict the mood for each song, I used multi-modal mood classification with neural networks, along with a mood map based on Russell's mood model and Last.fm user-generated tags. The color is calculated by mapping audio features to RGB and HSV codes.
Using this data, the software creates iteratively generated images for each song in the playlist, depicting the mood by using colors and abstract representations of the given text. VQGAN-CLIP is used to refine the visual elements and add a dynamic, textured quality to each image.
The final output is a video that visualizes the playlist, bringing the music to life with visuals that accurately represent the mood and color of each song. With Pixtape, I am able to create a cohesive and immersive experience for listeners, and showcase my skills in using cutting-edge technology to create beautiful, dynamic art.
Colab Notebook
Research Paper
Text Prompt Building Blocks
Generated Prompt Example
['abstract Fauvist watercolor painting of First Rain: 2 | blueviolet thistle: 0.9 | Relaxing: 1.8 | cynical: 0.8 | glowing neon: 0.9 | concert poster: 0.9', 'abstract Fauvist watercolor painting of Magical Mountains: 2 | blue black: 0.9 | Focused: 1.8 | sad: 0.8 | glowing neon: 0.9', 'abstract Fauvist watercolor painting of Oh, Lovely Appearance of Death: 2 | blue black: 0.9 | Relaxing: 1.8 | sad: 0.8 | glowing neon: 0.9', 'abstract Fauvist watercolor painting of Claudia, Wilhelm R And Me: 2 | royalblue black: 0.9 | Focused: 1.8 | calm: 0.8 | glowing neon: 0.9', 'abstract Fauvist watercolor painting of 33 “GOD”: 2 | slateblue black: 0.9 | Sad: 1.8 | dreamy: 0.8 | glowing neon: 0.9']