I asked AI to make a Music Video... the results are trippy
In this video, I utilized artificial intelligence to generate an animated music video for the song Canvas by Resonate. This tool allows anyone to generate beautiful images using only text as the input. My question was, what if I used song lyrics as input to the AI, can I make perfect music synchronized videos automatically with the push of a button? Let me know how you think the AI did in this visual interpretation of the song.
After getting caught up in the excitement around DALL·E2 (latest and greatest AI system, it’s INSANE), I searched for any way I could use similar image generation for music synchronization. Since DALL·E2 is not available to the public yet, my search led me to VQGAN CLIP (Vector Quantized Generative Adversarial Network and Contrastive Language–Image Pre-training), before settling more specifically on Disco Diffusion V5.2 Turbo. If you don’t know what any of these words or acronyms mean, don’t worry, I was just as confused when I first started learning about this technology. I believe we’re reaching a turning point where entire industries are about to shift in reaction to this new process (which is essentially magic!).
Important note:
While this AI is impressive, it still required additional input beyond just the song lyrics to achieve the music video I was looking for. For example, I added keyframes for camera motion throughout the generated world. These keyframes were manually synchronized to the beat by me. I also specified changes to the art style at different moments of the song. Since many of the lyrics are quite non-specific, even a human illustrator would have a hard time making visual representations. To make the lyrics more digestible by the AI, I sometimes modified the phrase to be more coherent, such as specifying a setting or atmosphere.
This was my first time working with DDV5, and I’m very happy with the results! There were many times where my jaw dropped upon seeing what the AI came up with. I haven’t felt this sense of wonder from technology since I first experienced a HD videogame as a child.
If you would like to learn more about how this video was made, try this yourself, or ask me any questions, I’ll post a more detailed explanation of how to get started on Patreon (link below). The post is free to the public, no need to pay. If you do want to support me and become a member that would be much appreciated, you’ll also automatically be entered into the end screen minigames where you earn points on each video and move up the leaderboard!
Join on Patreon to automatically have your name included in the next video:
Want to add lyrics and color beat blocks to your Disco Diffusion project like I did in my video? Here is my code:
My social media:
Twitter:
Discord:
Tiktok: @doodlechaos
Shorts Channel:
Instagram Shorts:
Email: contact@
While Disco Diffusion is based on the contributions of many, show some love to the two most prominent contributors:
Music:
[Indie Dance] - Rezonate - Canvas [Monstercat EP Release] :
1 view
151
30
4 days ago 00:15:39 1
(HD) #MillionaireAtChristmas #3rdEdition/#FamilyPictureEdition UK #DVDGameplay (#Season20242025) #4
4 days ago 00:01:18 1
[Rotten Tomatoes] Paddington Cast Reveals What’s Under Their Hats
4 days ago 00:03:24 8
[Plaga] Bandit in the Bazaar
4 days ago 00:14:57 1
(HD) #MillionaireAtChristmas #2ndEdition UK #DVDGameplay (#Season20242025) #4
4 days ago 00:04:16 1
[Movieclips] Motel Hell (9/10) Movie CLIP - Chainsaw Fight! (1980) HD
4 days ago 00:04:20 934
Woman with Facial Hair is Challenging Ideals of Beauty
4 days ago 00:01:43 540
Nosgov - Cost You an Arm (OFFICIAL VIDEO)
4 days ago 00:08:37 9
[Daily Dose Of Internet] Suspect Escapes Interrogation Room
4 days ago 00:00:14 3
Oversvømmelser har opslugt dusinvis af byer i det centrale og sydlige Irland
4 days ago 00:01:02 1
Обзор Тангенты для рации Baofeng
4 days ago 00:55:25 1
“Не внешнее избавление, но внутреннее исцеление“ Виктор Маршалко Церковь Христа Краснодар
4 days ago 00:03:02 1
Bir Aşk İki Hayat Дениз и Умут
4 days ago 01:06:35 1
Бизнес по-христиански. Панельная дискуссия
4 days ago 00:35:32 1.5K
Уровень Дипломатических Отношений России и США - Заместитель Посла В ООН Дмитрий Полянский | Скотт Риттер |
4 days ago 00:01:06 212
Отзыв о компании “АСК - СТРОЙ“
4 days ago 00:06:16 1
[OGxSTREET] CarX Street PC New Update🥵New Map is Fire !!
4 days ago 00:02:46 83
Сюкосева Екатерина, 13 лет. _Поехал казак на чужбину д...аск... пос (1). Спасский Верхнеуральского района Челябинской области (а
4 days ago 00:04:44 9
Simone Simons — In Love We Rust (Official Music Video)
5 days ago 00:00:17 6
Новый игровой компьютер “Орешник“ i5-13490F+RТХ4070SUРЕR
5 days ago 00:43:00 5
[meano the builder] The 900 ROCKET 4X4 | The OSMIUM - ANTI-SIEGE - FURNACE WALL | Rust CLAN Base Design 2024
5 days ago 00:11:56 124
[Ask History] Парусный боевой корабль - предел развития
5 days ago 00:16:12 1
(HD) #MillionaireAtChristmas #4thEdition/#FamilyEntertainmentEdition UK #DVDGameplay (#Season20242025) #3
5 days ago 00:00:23 4
Новый игровой компьютер “Моя Прелесть“
5 days ago 00:00:14 3
Новый игровой компьютер “Буревестник“ i7-12700К+RТХ4070SUРЕR