AI Brings an Image to Life

Using an image I generated in Midjourney a long time ago, I decided to test several AI programs to see if I could really bring the image to life. I also wanted to see if I could have the siren sing a song.

First, I needed a song. I tried using ChatGPT for lyrics, but I wasn’t thrilled with the results—very rhymey, very amateurish. However, it was useful for generating words associated with the sea and sirens, which was helpful. In the end, I had to write my own dark little song about luring victims into the sea. (I’ll put the lyrics at the bottom of the post.)

Next, I used my Suno account to create the song. I found out that the key to getting the sound I wanted was adding “soprano” to my prompt. Unfortunately, their AI associates “soprano” with opera rather than just a high singing voice, so it took a while to get a result I could use.

With the song ready, I tested out several AI programs: Immersity AI, LivePortrait, PixVerse, and Kling AI. The image I used is at the top of this page. I tried to use the same ridiculously simple prompt for all the programs: “A pretty woman sings. In the background ocean waves.”

The Best: Kling AI

By far, the best program was Kling AI. It not only brought my image to life but also lip-synced the Suno song I uploaded. The catch is Kling’s videos are only up to 10 seconds long, and lip-syncing only works if the character is facing the camera the entire time. So, in order to create my music video, I had to generate seven different 10-second clips. I also had to tweak my prompts to get her underwater or facing the camera for lip-syncing. The sync wasn’t perfect, but it was still very impressive. I used Microsoft Clipchamp to quickly throw together a music video, see the result below.

The Okayest: PixVerse

PixVerse was the most frustrating program to use. Most of my videos were ruined by the AI randomly generating horrible-looking hands flailing in the air. No prompt was able to stop the attack of the wild monstrous hands. Another issue: I couldn’t get the character to not sing or speak. PixVerse does not have the ability to create lip-syncing so I changed the prompt requesting the woman simply stand at the seashore and look around. I needed her to have a closed mouth so I could lip-sync the song later in LivePortrait, but it never worked. You can view the video below and see her chattering away. That said, PixVerse did have one redeeming quality: it stayed true to the original image better than the others. The AI really brought my siren to life, but I wasted so many prompts just to get one usable video.

The Disappointing: Immersity AI

Immersity AI didn’t impress me. It seems like you need to start with a photo that already has a 3D look in order to get a good result. While it technically made the character move as promised, it didn’t feel very life like. Also, unlike the previous two AI's the background will never move and that looks strange at the ocean when there should be crashing waves.

The Hilarious Failure: LivePortrait

LivePortrait was an absolute failure, but I’m humble enough to admit it could be operator error. I uploaded my image and a video of myself lip-syncing the lyrics, but the result were not great. I tried lip syncing multiple times, hoping for a better outcome, but no luck. The video below is the best I could do. However, since it’s an entirely free program, I can’t really complain. 

 Final Thoughts 
Overall, the programs were fun but I don’t think they are worth paying for at the moment. The failure rate is too high, meaning you waste multiple prompts just trying to get an okay result. Maybe I’ll revisit some of the programs in a year to see how far they’ve come.

The Song
Here is the full Siren Song. This is the version remastered by Suno’s V4 update. The voice sounds clearer and it stayed true to the original.


Siren Song Lyrics 

On the wind
a melody
Fall into the sea
The tide is calling you
to me

Soft seafoam upon your skin 
the water longs to pull you in
Drift 
far from the shore
return home never more

Listen softly to my song
Hear the sound
and let the waves pull you down

Salt and sorrow fill the air
The waves like fingers
through your hair
 
Close your eyes
fade into the deep
just breathe in
and sleep

Comments

Post a Comment

Popular posts from this blog

AI Sings Me a Song

Image
I fully believe technology will be the downfall of mankind and yet, it’s just so neat sometimes. Recently, I discovered Suno, an AI company that promises "a future where anyone can make music," and, of course, I had to try it out. Suno lets you type in your own lyrics, choose a musical style, and an AI algorithm will create a song for you in seconds. Best of all, they let you make several free prompts a day. Since I’m not looking to be part of any subscriptions, free sounds pretty good to me. I'm not into writing my own songs, but when I learned about Suno's AI, I immediately knew which song needed a musical update. Make Me a Channel of Your Peace , is a hymn based on the prayer of St. Francis. The original version, which you can easily find online, has a rather choppy and unattractive way about it. I’ve always felt that despite its beautiful words, the hymn just doesn't flow well and I wondered if the AI algorithm could help. Now, before anyone accuses me of...
Read more

Writing Your Own Children’s Book

Image
  During of the great Covid lockdown of 2020, I decided to write a children’s book to pass the time. My inspiration came from a childhood memory involving a confusing Christmas gift from my grandmother. She had gifted me a bicycle tire pump. As I tore through the wrapping paper I got really excited believing there must also be a bicycle hidden somewhere nearby. I searched the house high and low but it turned out it was all just a misunderstanding. Grandma was simply unaware I didn’t have a bicycle, and had no need for a tire pump. I’m sure many can recall a time of receiving a similar mismatched gift during birthdays or holidays. I thought it would be fun to write a story where all those strange gifts turn out to be unexpectedly useful after all. The story came together relatively quickly. At first, I just wrote a few ideas down and would work on it every once and a while. It was only after I figured out what each of the five unique gifts would be that I was able to really sit down...
Read more

Of Bunnies and Hens

Image
Recently, I’ve been crocheting a few gifts, and I decided to get a little creative. Instead of using different colored yarns, I experimented with fabric paint to bring my amigurumi to life. I was really happy with the results and thought I’d share my experience. A coworker sadly lost her adorable pet rabbit, and I wanted do something special for her. I already had a crochet pattern for a rabbit , but this time I decided to sculpt the face with a needle and thread. The outcome was a much cuter and more realistic look than last time. Here is a picture of the rabbit unpainted: I lightly painted the rabbit, giving it a soft watercolor look. The photo doesn't quite capture how nice the greys with a touch of brown looks in real life. I hope it brought her a little comfort during a tough time.   Later, a family member requested a crocheted hen for their birthday. I found an adorable pattern called “Flora the Hen” from an Etsy shop called Airali. The pattern was well written, tho...
Read more