BLISSathon Google: Creating Real-Time AI Music Visualizations
Published:
BLISSathon Google: Creating Real-Time AI Music Visualizations
I’m excited to share a project that our team Justus Westerhoff, Hubert Tomaszczak, Coco Bögel, Noah König and I have recently accomplished - BLISSathon Google. During that time we looked into how AI may improve music and art. We set out on a mission to build real-time music visualizations, inspired by the old iTunes visualizer. In the absence of a live band, we decided to create real-time music videos.
Technical Implementation
Discovering StreamDiffusion enabled us to realize our vision. Our journey began with utilizing Gemini to create themes for various songs, focusing on a variety of topics, primarily hackathons and computer science. Trying to merge them with distinct genres and instrumental styles was a challenge. Thanks to you guys it was successfully accomplished.
The Process
Next, we have set on using Suno AI to create the music and lyrics. Gemini then converted the music to text with timestamps, which we utilized to generate 10-second visual parts. We generated a smooth 30FPS animation by linearly interpolating the Autoencoder’s latent vectors. StreamDiffusion supported us in merging these photos into a unified film that synchronized flawlessly with the sounds.
Key Technologies Used
- Gemini AI: Theme generation and music-to-text conversion
- Suno AI: Music and lyrics creation
- StreamDiffusion: Real-time image generation and video synthesis
- Autoencoder Latent Vectors: Smooth animation interpolation
- 30FPS Animation: High-quality video output
Project Outcome
A great heads-up to BLISS e.V. and Google, for hosting this event and for providing the tools, space, and recommendations during their Hackathon. The event provided us with an excellent opportunity to bring our ideas to life and to the needed practice in programming. After around 8 hours of intense work, we were overjoyed to win the Jury Challenge. It was awesome to see our hard work appreciated.
Impact and Future
I look forward to more projects like this in the future! BLISS e.V. - you are just amazing in what you are doing. Your Speaker Series and Hackathons have given me so much inspiration and encouragement to continue working on my own AI projects and bringing them to life.
Team: Justus Westerhoff, Noah König, Coco Bögel, Hubert Tomaszczak - Bravo! Tausend Dank, dass Sie uns stets gerne weiterhelfen. Super Zusammenarbeit!
#hackathon #bliss #blissathon #google #ai #berlin
Original LinkedIn Post: View the original post and engagement on LinkedIn
About the Author: David Mkhitaryan is an aspiring AI/ML developer with practical experience in LLM Fine-Tuning, RAG systems, and data analysis. Currently working as a Venture Capital Analyst at Innovis VC while pursuing studies in Industrial Engineering at Technical University of Berlin.