Runway AI Gen-2 is one of the most exciting AI-powered tools that I've seen thus far. I personally believe that it will play a large role in how content is made in the future.
Although I know that Midjourney plans (and if you're a loyal reader you know how big of a Midjourney fan I am) to release their own text-to-video model in the future, and I'm excited for it, I believe that it's important for models from multiple teams to coexist. People should have a choice regarding which tool they'll use.
What Runway has managed to do with Gen-2 is nothing short of amazing.
While some people are saying that the video quality is not that impressive, I urge you to remember that developing novel tools like this takes time. Remember what type of results you got from Midjourney when it was first launched?
It's important to think about the potential of a product when it is in its very early stages. Right now is the perfect time to start experimenting with different prompts so you can be ahead of most people when they become impressed enough to start using the Runway AI model to create content.
And there's certainly a lot of content to be made with a model like this. You can use it to make short films, music videos, social media content, ads, and so on.
Today, I want to introduce you to the power of Runway AI. I'll take a few minutes to introduce you to the product and the company behind it, and then I'll share some fun prompt examples and the generated outputs.
What Is Runway AI?
Have you ever heard of a popular text-to-image model called Stable Diffusion? It became a hit since it was released publicly and made to run on most consumer hardware, unlike most other models that are accessible only through cloud services.
The reason why I'm mentioning Stable Diffusion is because the Runway research company co-created that very model. This is a company that's been in the AI industry since 2018 and they're one of the brightest teams in the space.
We live in a time where video content is king. The web went through multiple phases since its inception. It went from text to images to video. Although AI-generated video content is in its early stages, there's clearly a big demand for it.
The core product developed by the Runway company is a video editing platform that enables you to create videos from either text or images.
The model this company developed utilizes both artificial intelligence and machine learning algorithms.
There are also some useful video editing tools available on the Runway AI platform. Some of these tools can help you with motion tracking, scene detection, color grading, inpainting, and more. There are many cool features that you can use when you sign up for the platform.
The first iteration of the platform was released in February 2023 and became known as Gen-1. It included a model that enables users to change existing videos by applying certain effects. It didn't take long for the company to release Runway Gen-2. It became available in March 2023 and instantly received praise.
There is a lot to be covered here in regards to Runway, but I'm not going to make this a long article. Instead, I'm going to focus on the basics. But you can definitely expect a lot more Runway-related content from me in the near future.
How to Sign Up for Runway AI?
It's easy to sign up for the Runway AI platform. All you have to do is navigate to the official Runway website and press the 'Sign Up' button in the upper right corner of the home page.
Fill out the simple form and you'll be set up with a free account. With this account, you'll get 125 credits. Credits are essentially tokens that you spend to generate videos. With the free account, you can test the capabilities of the Runway AI models.
After you sign up, you'll see multiple tools available as shown in the image above. As you scroll down the page, you will also see some helpful tutorials that will teach you how to use both Gen-1 and Gen-2 models.
One thing I really like about the Runway AI platform is that there is also an iOS app available. I love using this app on the go. I will write a detailed article on how to use the app in the future.
How Much Does Runway AI Cost?
I've been hearing complaints on social media that Runway AI is too expensive. I understand that many people would like to use this platform but can't afford the extra monthly bill. But I also fully understand how much money it costs to keep a platform like this running where people are able to generate video content quickly.
It's not my goal to share my personal opinions on price. My aim is to simply share the information I deem to be important to my readers. With that being said, here are the different price plans available on the Runway platform:
- The standard plan costs $12 per month and gives you full access to the platform, along with 625 credits for each month. Each second of video generated in Runway Gen-2 costs 5 credits. With the standard plan, you can generate 125 seconds of Gen-2 clips, or roughly 31 standard videos.
- The pro plan costs $28 per month and comes with 2250 credits each month, which means you get 450 seconds of video generation. That equals roughly 112 standard videos each month. You can also buy additional credits when needed.
- The ultimate plan gives you the option for unlimited video generations. You get 2250 credits each month to use in Credits Mode. When you use all of your credits, you can continue generating as many videos as you want at a relaxed rate.
There is also an enterprise plan which is aimed at companies, but there is no fixed rate for this plan. You have to contact the Runway team to work out a specific plan.
What I like about Runway is that there is the basic plan, which is free and enables users to try out the platform. With the free plan, you get 125 credits that you can use to generate several videos in Gen-2.
How to Use Runway AI?
I decided to start a small side project several days ago. It's a Twitter/X account called Prompt Beam where I share some of my AI-generated images and videos. I'm still in the process of thinking about which direction I'm going to take with this account, but you'll definitely see a lot of interesting content on that page.
I'm going to be posting all the Runway AI videos I make on this account, and I'll share them on this website in all Runway-related articles. I'm writing this so that you don't get confused when you see the tweets below.
Anyway, I'm going to show you today how you can use both text and images to create videos in Runway Gen-2. Since this is a beginner's guide, I'll keep the prompts relatively simple. Let's start with an easy text-to-video prompt.
Runway AI Text to Video Prompts
To begin, you have to navigate to your dashboard of the official Runway website and click on the 'Text to Video' button. After you do that, you'll be taken to a page where you can write a text prompt.
One of the first prompts I wrote in Runway Gen-2 goes like this:
prompt: wide cinematic shot, astronauts walking on a strange alien planet, peculiar landscape, moody
Here's the result I got from that prompt.
If this is your first time seeing the videos Runway AI can produce and don't think they're good, I urge you to take a step back and think about the potential quality in the future. People are already making ads, music videos, and other types of content with this AI model.
As long as you develop your skills early, you can become an expert at this stuff and that might prove to be quite useful in the future.
Since this is a beginner's guide, I'll stick to the basics. But you can expect more advanced articles on Runway Gen-2 in the near future, including how to make long, coherent bodies of work with sound.
Let's try making another short video using a relatively simple prompt.
prompt: mysterious person dressed in a cloak walking through the desert alone
I like the way this video turned out. You could say that the motion of this person is not great, but I have no intention of nitpicking when it comes to Runway Gen-2. The fact that it's even possible to create a video like this purely from text is mindblowing to me.
At the time of writing this article, there is definitely a lack of text-to-video AI tools available. Meanwhile, I've seen text-to-image models popping up left and right in recent months.
I'm not an expert at the technical side of AI models. I couldn't tell you exactly how they're made. I'm confident in my research skills and could probably learn and relay that information, but that's not what I'm here to do. I want to inspire other AI enthusiasts to use a variety of tools because I firmly believe that this technology will play an important role in shaping the future.
You could probably tell how passionate I am about this, but I don't want to get carried away. Instead, I'll get back to prompting. Although you can definitely expect a ton more text to video Runway prompts for me in the future, I want to take a few minutes to show you how the image to video tool works.
Runway AI Image to Video Prompts
When you go to your account dashboard on the official Runway website, you'll see the image-to-video button right next to the text-to-video one. Press the button and you'll be taken to a page where you can upload an image and create a video out of it.
I've spent the past several months making thousands of images in Midjourney, so I have a lot of content to choose from. I'm a person who can spend the next 10 hours looking for the perfect image, so I just picked one completely at random to save time.
The one I picked is featured in my article about Midjourney prompts for digital avatars, and it looks like this.
Now bear in mind that I don't have to do anything except upload this image and press the 'Generate' button on the Runway platform. Here's the video I got from the image above.
Once again, there are clear flaws here. It appears as if the woman has four arms. I hear you, it can be better. But you need to have patience. And maybe you already do, but I'm writing this to encourage as many people as possible to experiment with tools like this.
If you don't follow the news surrounding Runway AI closely, you've probably missed out on an announcement from August 10 when the team issued an update that enabled users to extend their Gen-2 videos up to 18 seconds. This is a huge deal, as the previous limit was only 4 seconds.
When you generate a video from the initial prompt, it will still be 4 seconds in the beginning but then you'll be able to extend it by another 4 seconds and repeat the process until you reach the 18-second limit. Let's test this feature out on a new prompt.
This time, I used an image that is featured in my article about Midjourney prompts for anime.
After the video is fully generated, you'll see a button that says 'Extend 4s', which you can press to extend the clip. I'm going to extend this video only once to show you how it works, but remember that you can repeat the process multiple times for each video.
I've marked where you can find the button for extending videos in the image above. When I extended the clip, I got an 8-second video, which you can check out below.
This video looks absolutely amazing. I was pleasantly surprised to see the female character turn into a male character. I also appreciate the change in art style. These are the two reasons why I named the video duality.
Now that you've learned how to make videos from images, it's time to embark on the final quest for today - using both text and images to generate videos.
Runway AI Text + Image to Video Prompts
You can use a combination of both text and images to generate new videos in Runway Gen-2. Before you start doing this, there is one extremely helpful piece of advice I want to share.
Although you can get great results by adding context to an image through text, you should do your best to ensure your words align well with the image you uploaded. The AI model will generate a video regardless of whether it's aligned or not, but you shouldn't expect the video to match the image closely if the text is much different from the image.
This might sound a bit too confusing, so I'll just show you an example.
The images I've used for this article so far have been illustrations. This time, I wanted to use a photorealistic image that I created in Midjourney. This image was featured in my article about Midjourney prompts for nature. You can check it out below.
Before I share what type of text I added along with this image in my prompt, I have to first teach you how to write these combo prompts.
The first step is to press the 'Image to Video' button and upload an image that you want to use. Once you do that, you can press the 'Text' button on the left. This will provide you with the option to include the text that you want.
Like I previously said, the text should at least somewhat align with the image. I decided to write 'aerial drone footage' in the text part of my prompt. Here's the result.
What's interesting here is that the initial image has been altered in a peculiar way. The landscape looks different but if I didn't have both the image and the video to compare side by side but was instead going on memory, I would say that they look basically the same.
I want to do one more combo prompt and that's going to be it for this article as far as prompts go. For the next video, I'm going to be using an image featured in my Midjourey aesthetics guide.
This is an image that I made while I was experimenting with darkcore aesthetics in Midjourney. Essentially, I wanted to create an image that looked like it was a shot from a slasher horror movie. And I think I did a pretty good job. But it's only an image.
With Runway, I knew I could bring this image to life with the right text prompt. I decided to include only three keywords in the text prompt - panic, breathless, scary. Here's the result.
It's important not to have your expectations too high when you're doing combo prompts. The generated output will hardly ever look like a clear continuation of the initial image. However, if you keep the text prompt simple and aligned with the image, it will lead to a great result.
What I like here is that the AI model used the initial image as inspiration to a great extent. The women look incredibly similar, and they even have blood on their cheeks in both instances. I'm very happy with this video.
It's no accident that both of the images I used for these combo prompts were photorealistic. From what I've seen and experienced, writing text + image prompts gives the best results when the images are photorealistic. I wouldn't be able to get results like this if I were to use illustrations.
Rating the Results
I want to take a quick minute to discuss how important it is to rate the results of each video you generate. By doing this, you help train the AI model, which benefits all other users on the platform.
The AI model is trained through reinforcement learning. Every time you rate the generated output, it has an important impact on the model.
If you're new to Runway or AI-powered tools in general, you should make it a habit to take a second to rate each generated video. Give an honest rating, even if you think the output is awful.
By showing the model what is good and what isn't, it will understand better what to generate in the future. You may think that your votes don't have any impact, but that simply isn't true.
As AI enthusiasts, we should all strive to not only seek help from artificial intelligence but to offer it as well. In my opinion, this is a two-way street. And as a community, we should all stick together and do what's good for each other.
Final Thoughts
Runway Gen-2 is an incredibly powerful AI model that can transform either only text or a combination of text and images to create videos. I want to reiterate once more how important video content is in today's world.
One of the most frequent comments people make about Gen-2 is that the quality of the videos is not that good.
Do I personally think that the quality could be better? Absolutely. Do I think it will be improved with future iterations of the AI model? Of course.
Things take time to develop, and I love that many companies in the AI industry are taking this approach where they improve their product at an extremely high pace. I have no doubt that we'll see Gen-3 get released in the near future and it will blow everybody away.
AI-generated content is becoming more popular with each passing day. It's up to the enthusiasts to learn how to make the best content with the help of artificial intelligence. With that said, I look forward to exploring Runway AI in many more articles to come.
