If you had told me a few years ago that I would be able to transform a line of text into a video, I wouldn't have believed it. But here we are. The Runway Research company announced its Gen-2 model in March 2023, and it became the first publicly available text-to-video AI model.
I started to dabble in artificial intelligence through large language models like ChatGPT. After that, I shifted my focus to popular text-to-image AI models like Midjourney. These days, I spend a lot of time researching and experimenting with this brilliant text-to-video model.
People have already found many use cases for Runway Gen-2, such as making music videos and commercials. If you've never used Runway AI before, I suggest you read my Gen-2 beginner's guide. It has more than enough information to get you started with this model.
Today, I want to share multiple Runway Gen-2 text-to-video prompt examples. These examples encompass different styles and aesthetics, but you can expect many more text-to-video prompt articles from me in the very near future.
How to Write Runway Gen-2 Text to Video Prompts?
There are many strategies that you can use to write good Runway Gen-2 text to video prompts. The biggest piece of advice I can give you is to experiment with different methods to see what works and then develop your own style of expression. This will be much easier to do if you have had experience using generative AI models before.
One thing I will mention right away is that a prompt in Runway Gen-2 can contain up to 320 characters. This is more than enough to write a good, detailed prompt in my opinion.
If you don't know where to start with prompting, I'll give you some excellent pointers. Remember that you're creating videos in Runway, so you'll need to keep several things in mind when you're describing the scene you want to make.
- The first thing to consider in your prompt is what the idea for the scene is about. For example, it can be a scene where a car drives down an empty highway. It can be a rocket taking off. Develop the idea for the scene before anything else.
- Once you have imagined the scene, think about whether you want to include one or more characters. Essentially, you'll want to consider whether a subject can make the scene better.
- If there is a subject in the shot, think about whether you want to add movement to the video. Do you want the subject to be walking/running/swimming or sitting down? Sometimes, it's also a great idea to mention eye movement. Maybe the main character should be looking directly into the "camera".
- The next thing you want to pay attention to is what type of style or aesthetic you want applied to the video. For instance, it can be a film noir aesthetic or a high fantasy style.
- Another really important thing is the setting. In what type of environment is the scene set in? This will play a huge role in how the final video turns out.
- Finally, the last thing you might want to consider including in your prompt is lighting. Think about whether a certain type of lighting would make the scene stand out.
Those are some of the useful tips I'd give to anybody who is interested in creating content with Runway Gen-2, but especially new users who may not be used to writing prompts.
If you have dedicated some portion of your time using text to image models, it will be easier for you to become comfortable with Runway Gen-2. There are some big differences in how you should write prompts for text to image and text to video AI models, but it'll be much easier for you to learn how to make good videos if you have prior experience with any generative AI model.
Runway Gen-2 Text to Video Prompt Examples
I could write a much more detailed explanation of how to write good prompts for Runway Gen-2, but I personally feel like the best way for people to learn is through prompt examples.
Prompt examples are what I like to focus on and share with people when I'm creating content. That's why I'm excited about this part of the article because I'll be sharing some videos that I made using Runway Gen-2.
Here are some of my favorite Runway Gen-2 prompt examples that I've written and used to make videos so far.
prompt #1: old man sitting in the middle of a field holding a book and looking directly at the camera, post-apocalyptic environment, low key lighting, darkcore aesthetic
I started writing this prompt and ultimately realized how inspired I was by the Book of Eli film when I finished writing it. Low key lighting means that there is only a single strong key light source. You can usually see this in mystery, horror, and thriller movies.
I like how the video turned out. I will say that I wished there were more post-apocalyptic details in the background. But I think that it's going to be possible to add significant detail to each video in the next iterations of this model when the Runway research company is finished building them.
prompt #2: 1950s aesthetic, high key lighting, 1950s fashion model walking down an empty beach on a sunny day, walking away from the camera, full color, photoshoot
It took many tweaks to get this prompt right. I started out wanting to do a scene where people were playing beach volleyball, but I ultimately settled on the prompt you see above. Here's how the clip turned out.
Proper movement is a really tough problem to solve when it comes to an AI model like this. The movement here is really good considering the current capabilities of Runway Gen-2. I also had to add the 'full color' keyword here because before I did the generated videos were all black and white.
prompt #3: cinematic shot, suspense, man walking towards a glowing sacred altar, mystery ambiance, dim lighting
When you're writing a prompt for Runway Gen-2, the best thing you can do is click on the "Free Preview" button before you generate a video. This will give you multiple options so you know what to expect when you're creating a video and it will help you be more strategic when you're spending credits.
This is an example of great subject movement in Gen-2. It's difficult to predict when the Runway AI model will generate something that looks natural and when it won't. I want to mention that 'cinematic shot' is one of my favorite keywords in Runway.
prompt #4: skyline of a futuristic dystopian city, bright neon lights shining, sci-fi film aesthetic
This is quite a simple prompt that can produce great results. Sometimes, you get the best videos when you don't include any human subjects in the video. It'll likely take some time before the Runway AI model learns how to generate human movement perfectly.
I really like the animation here. I have no idea what's in the upper left corner at the beginning of the clip, but it looks like a spaceship that instantly disappears into thin air. That part wasn't in the prompt but I have no objections to its inclusion in the generated video.
prompt #5: aerial drone view, a mighty avalanche, scary, action-packed scene
This is another relatively simple prompt that I wrote to see how the AI model would present the power of nature. I find avalanches quite scary. In fact, one of my biggest fears is to get caught in one because I know my chances of survival are not good.
I've tried multiple variations of this prompt and generated quite a few videos, but this is the best result I got. The avalanche could have been better, but it's still a really good shot. I like to use the 'aerial drone view' to get shots like this in Runway Gen-2.
prompt #6: a magical forest during the day, lush greenery, high key lighting
High key lighting is a term used to describe a particular style of lighting that creates a bright scene. This style is often used in music videos, commercials, but it's also common in a lot of movies.
You could make a cool short film using Runway Gen-2 with some effort. I believe that it's worth doing and is something I plan to do as a special project in the future. But before I do that, I first want to write multiple guides to help you better understand this amazing text-to-video model.
prompt #7: high fantasy setting, medieval castle, fantasy film aesthetic, soft lighting
Soft lighting is often used in both romance and fantasy films because it's used to convey happiness and hope. I wanted to create a clip of a dragon flying over a medieval kingdom and I ran through at least a hundred previews and generated multiple videos before giving up on that idea. After all, that's a very specific request and the movement of a dragon is quite complex.
I like the lighting here and I love the aesthetic. It kind of reminds me of Disney movies. I expected something a bit more serious, but this works incredibly well too. And that's about it when it comes to Runway Gen-2 prompt examples for today, but you can expect more from me soon.
Final Thoughts
I am really impressed by the capabilities of the Runway Gen-2 text to video model. It takes an incredibly short amount of time to create a video using only a text description.
There are times that you might get unexpected results that don't really match your prompt or the videos are simply different from how you envisioned them. When this happens, all you have to do is slightly edit the prompt.
Don't expect to generate amazing videos on your first try. It usually requires at least some tweaking to get it right. But I would argue that you learn efficiently when you're doing these tweaks because that's how you learn what works and what doesn't.
