Alchemy with AI-Generated Animation and Salsa

I decided to conduct a little experimentation with a relatively recent extension in Stable Diffusion (SD) that allows the creator to generate animated videos. I had briefly played with AnimateDiff (AD) about a month or so ago, but all I got was a distorted animation that looked like a ‘big ball of wibbly-wobbly, timey-wimey stuff’. This meant I would have to do some some technical maneuvering behind the scenes with files and script, so my eyes quickly rolled to the back of my head, and I left it alone. I don’t know what demon possessed me the other day, but I decided to put on my big boy pants and get a handle on AD.

So I fired up SD, pulled some levers, hit some buttons, turned some wheels, and slapped some hardware until I got AD working and created a rudimentary gif. I cannot call it an animation because it’s a bunch of random images, so perhaps it’s an animation in the general sense. The prompt was supposed to generate a blooming flower

As you can see, strictly speaking, there’s no blooming happening anywhere. Furthermore, sometimes AD would chug along just fine, and other times it would crash. I was mystified about what was happening. So again I pulled some levers, hit some buttons, turned some wheels, and slapped some hardware. I got it working with a new set of parameters and created the following animations of a leaf blowing in the wind (shown in order of creation)

Picasso and Max Headroom would be impressed, but it was not what I was looking for. So guess what? I pulled some levers, hit some buttons, turned some wheels, and slapped some hardware. I was getting really intense, like I usually do when I get the hamster spinning in the mind-wheel. This was going to happen by sheer force of will if necessary.

I once again prompted SD for a leaf blowing in the wind, and this was the result.

Violin! I mean, voilà!

As you can see, the animation is smooth and consistent across frames. I used all the bare minimum settings because I did not want to mess up anything and give myself a technical headache. I’m a content creator, Jim, not an engineer! I cannot tell you how happy I was when I saw that animation. My nipples perked up, and I Doordashed some tacos. Then all re-energized with salsa and guacamole, I played around with AD some more and created the following animations:

😍

Not too shabby, eh? I’m looking forward to more experimentation and hopefully even more tacos! I'll leave you with the last piece that shows a delightfully complex animation with parallax! (though someone should probably call the firefighters). Thank you!