Friday Fun: AI music videos are here

This modern era of GPU-accelerated AI applications have their benefits. Pulling useful information out of mountains of raw data, alerting users to driving hazards, or just keeping an eye on bee populations are all helpful.

Lately there has been a rise in attempts at producing (or should that be curating?) with help of AI works of art out of carefully sculpted inputs.

One such AI art project is midjourney: That bot takes some textual input, then “dreams” with it, producing sometime uncanny, often downright disturbing images.

Firdt there were just images. A new craze is to create music videos using Midjourney and simikar AI tools. The idea is to feed song lyrics in short pieces to the AI tool, let the AI to generate images that are combined to make a music video. Here are some examples of such music videos made to well known songs:

Stairway to Heaven – Led Zeppelin – But the lyrics are Ai generated images
https://www.youtube.com/watch?v=ZCKdPhepB1s

More videos:

Immigrant Song by Led Zeppelin – Every lyric is an AI generated image
https://www.youtube.com/watch?v=iUi2K6cMNAk

Hotel California – Eagles – But the lyrics are Ai generated images
https://www.youtube.com/watch?v=bulNXhYXgFI

Enter Sandman – Metallica – But the lyrics are Ai generated images
https://www.youtube.com/watch?v=KVj_AEhpVbA

All Star – But, every lyric is an AI generated Image
https://www.youtube.com/watch?v=MuAJKB9mCfc

Don’t Fear The Reaper – Blue Oyster Cult – But the lyrics are Ai generated images
https://www.youtube.com/watch?v=rhf4DZL0kpM

Black Hole Sun – But it’s an a.i. Visual Lyric Video
https://www.youtube.com/watch?v=o5xtrK0iF6M

Killer Queen – But every lyric is an AI generated image
https://www.youtube.com/watch?v=Ei3Mg7ajMP4

Don’t Stop Me Now | But Every Lyric is an AI Generated Image
https://www.youtube.com/watch?v=exeo1MnsG0U

Run to the Hills – But every lyric is an AI generated image
https://www.youtube.com/watch?v=8DwGAE7Buf4

Hail to the King – But every lyric is an AI generated image
https://www.youtube.com/watch?v=ZfUvBW-1mx4

Billie Jean but every lyric is an AI generated image
https://www.youtube.com/watch?v=8MlJ2USBiCc

(Mid)Journey – Don’t Stop Believin’ but the lyrics are AI generated images
https://www.youtube.com/watch?v=GpgZlmEujUs

The Sound of Silence – a.i. Generated Visual Lyric Video
https://www.youtube.com/watch?v=uiEDsquddX0

Mr. Sandman, but every verse is an AI generated image
https://www.youtube.com/watch?v=86R4KneBlhc

6 Comments

  1. Tomi Engdahl says:

    John Herrman / New York Magazine:
    An in-depth look at AI image-creation tools like Imagen, DALL-E 2, and Midjourney; Midjourney users are making concept art for clients and using stock art less — How we work — even think — changes when we can instantly command convincing images into existence.

    AI Art Is Here and the World Is Already Different How we work — even think — changes when we can instantly command convincing images into existence.
    https://nymag.com/intelligencer/2022/09/ai-art-is-here-and-the-world-is-already-different.html

    Artificial-intelligence experts are excited about the progress of the past few years. You can tell! They’ve been telling reporters things like “Everything’s in bloom,” “Billions of lives will be affected,” and “I know a person when I talk to it — it doesn’t matter whether they have a brain made of meat in their head.”

    We don’t have to take their word for it, though. Recently, AI-powered tools have been making themselves known directly to the public, flooding our social feeds with bizarre and shocking and often very funny machine-generated content. OpenAI’s GPT-3 took simple text prompts — to write a news article about AI or to imagine a rose ceremony from The Bachelor in Middle English — and produced convincing results.

    Deepfakes graduated from a looming threat to something an enterprising teenager can put together for a TikTok, and chatbots are occasionally sending their creators into crisis.

    More widespread, and probably most evocative of a creative artificial intelligence, is the new crop of image-creation tools, including DALL-E, Imagen, Craiyon, and Midjourney, which all do versions of the same thing. You ask them to render something. Then, with models trained on vast sets of images gathered from around the web and elsewhere, they try — “Bart Simpson in the style of Soviet statuary”; “goldendoodle megafauna in the streets of Chelsea”; “a spaghetti dinner in hell”; “a logo for a carpet-cleaning company, blue and red, round”; “the meaning of life.”

    Through a million posts and memes, these tools have become the new face of AI.

    This flood of machine-generated media has already altered the discourse around AI for the better, probably, though it couldn’t have been much worse. In contrast with the glib intra-VC debate about avoiding human enslavement by a future superintelligence, discussions about image-generation technology have been driven by users and artists and focus on labor, intellectual property, AI bias, and the ethics of artistic borrowing and reproduction.

    Reply
  2. Tomi Engdahl says:

    Kyle Wiggers / TechCrunch:
    OpenAI starts letting DALL-E 2 customers upload and edit people’s faces, after first banning photos depicting real people such as celebrities and public figures

    OpenAI begins allowing users to edit faces with DALL-E 2
    https://techcrunch.com/2022/09/19/openai-begins-allowing-users-to-edit-faces-with-dall-e-2/

    Reply
  3. krogereschedule says:

    Awesome and interesting article. Great things you’ve always shared with us. Thanks. Just continue composing this kind of post.

    Reply
  4. krogereschedule says:

    Thanks a lot for the reply. I will try the suggested solution from your link.

    Reply
  5. Tomi Engdahl says:

    AI Dreaming Of Time Travel
    https://hackaday.com/2022/09/28/ai-dreaming-of-time-travel/

    We love the intersection between art and technology, and a video made by an AI (Stable Diffusion) imagining a journey through time (Nitter) is a lovely example. The project is relatively straightforward, but as with most art projects, there were endless hours of [Xander Steenbrugge] tweaking and playing with different parts of the process until it was just how he liked it. He mentions trying thousands of different prompts and seeds — an example of one of the prompts is “a small tribal village with huts.” In the video, each prompt got 72 frames, slowly increasing in strength and then decreasing as the following prompt came along.

    https://twitter.com/xsteenbrugge/status/1558508866463219712

    Voyage through Time – a Generative AI journey
    https://www.youtube.com/watch?v=Bo3VZCjDhGI&t=146s

    Reply
  6. Tomi Engdahl says:

    James Vincent / The Verge:
    Meta details its text-to-video AI generator, Make-A-Video, which can produce up to five-second videos without audio; Meta is not giving access to the AI model

    Meta’s new text-to-video AI generator is like DALL-E for video
    https://www.theverge.com/2022/9/29/23378210/meta-text-to-video-ai-generation-make-a-video-model-dall-e?scrolla=5eb6d68b7fedc32c19ef33b4

    AI text-to-image generators have been making headlines in recent months, but researchers are already moving on to the next frontier: AI text-to-video generators.

    A team of machine learning engineers from Facebook’s parent company Meta has unveiled a new system called Make-A-Video. As the name suggests, this AI model allows users to type in a rough description of a scene, and it will generate a short video matching their text. The videos are clearly artificial, with blurred subjects and distorted animation, but still represent a significant development in the field of AI content generation.

    Introducing Make-A-Video: An AI system that generates videos from text
    https://ai.facebook.com/blog/generative-ai-text-to-video/

    Today, we’re announcing Make-A-Video, a new AI system that lets people turn text prompts into brief, high-quality video clips. Make-A-Video builds on Meta AI’s recent progress in generative technology research and has the potential to open new opportunities for creators and artists. The system learns what the world looks like from paired text-image data and how the world moves from video footage with no associated text. As part of our continued commitment to open science, we’re sharing details in a research paper and plan to release a demo experience.

    Generative AI research is pushing creative expression forward by giving people tools to quickly and easily create new content. With just a few words or lines of text, Make-A-Video can bring imagination to life and create one-of-a-kind videos full of vivid colors, characters, and landscapes. The system can also create videos from images or take existing videos and create new ones that are similar.

    Make-A-Video follows our announcement earlier this year of Make-A-Scene, a multimodal generative AI method that gives people more control over the AI generated content they create. With Make-A-Scene, we demonstrated how people can create photorealistic illustrations and storybook-quality art using words, lines of text, and freeform sketches.

    https://makeavideo.studio/

    Reply

Leave a Comment

Your email address will not be published. Required fields are marked *

*

*