Add Stunning AI B-Roll toVideos for Free: A Complete Tutorial | by Anil Chandra Naidu Matcha | Jul, 2024

In within the current day’s data, I’ll current you simple strategies to seamlessly use AI know-how to quickly uncover and incorporate the proper B-roll, saving you time and enhancing your motion pictures effortlessly. First let’s understand what’s a B-roll.

B-roll refers to supplementary footage that is intercut with the first footage in a video. It enhances the storytelling by providing additional context, supporting the narrative, or illustrating elements being made inside the primary footage (typically often called A-roll). B-roll can embrace numerous clips, equal to cutaways, transitions, or background footage, that help to create a further partaking and visually fascinating video. For content material materials creators, environment friendly use of B-roll can significantly elevate the manufacturing top quality, making motion pictures further dynamic and engaging for the viewers.

The strategy of determining B-roll for a video consists of each taking footage them individually or deciding on associated footage from on-line sources equal to stock suppliers like Pexels and so forth. manually. So let’s now attempt to automate this course of

This AI B-Roll generator is an open-source software program on the market on GitHub. It makes use of AI to automate the strategy of discovering and integrating B-roll footage into your motion pictures. Beneath is the workflow we observe in order so as to add b-roll to our video

1. Generate captions for enter video
2. Set up key phrases which symbolize these captions

• Occasion : AI is used to automate many of the human duties -> Automation

3. Fetch pexels motion pictures for these key phrases to utilize as b-roll motion pictures

4. Sew collectively the b-roll motion pictures with distinctive video

Let’s do these step-by-step

First, you’ll need to put within the required dependencies. These embrace:

1. Whisper Library: Used to generate captions to your audio.
2. Pytube: Used to acquire motion pictures from YouTube.
3. OpenAI: Used for AI-related duties, along with key phrase know-how.
4. MoviePy: Used for video enhancing.

We start by downloading the enter video using Pytube. Chances are you’ll each enter a YouTube URL or add a video on to Google Colab. We are going to modify video_url from the beneath code to acquire any video of our various. The video shall be acquire to file named video.mp4

from pytube import YouTube
import os
# Function to acquire a YouTube video
def download_youtube_video(url, output_path='.', filename=None):
try:
# Create a YouTube object
yt = YouTube(url)
# Get the perfect choice stream on the market
video_stream = yt.streams.get_highest_resolution()
# Receive the video with the specified filename
downloaded_file_path = video_stream.acquire(output_path=output_path, filename=filename)
# If a filename is specified, rename the file
if filename:
base, ext = os.path.splitext(downloaded_file_path)
new_file_path = os.path.be part of(output_path, f"{filename}")
os.rename(downloaded_file_path, new_file_path)
print(f"Downloaded and renamed to: {new_file_path}")
else:
print(f"Downloaded: {downloaded_file_path}")
apart from Exception as e:
print(f"An error occurred: {e}")
# Occasion utilization
if __name__ == "__main__":
# URL of the YouTube video to be downloaded
video_url = 'https://www.youtube.com/watch?v=8ZyShHwF_g0'
# Output path the place the video shall be saved
output_path = '.'
# Receive the video
download_youtube_video(video_url, output_path, filename="video.mp4")

Subsequent, we extract the audio from the video using FFmpeg and transfer it to the Whisper library to generate captions using the beneath command. If we’re working the code in Google colab, ffmpeg is put in by default and subsequently there is no such thing as a such factor as a need to put in it individually. We then load whisper medium and run by the use of our audio to generate the captions

!ffmpeg -i video.mp4 -ab 160k -ac 2 -ar 44100 -vn audio.wav
import whisper
# Load the model
model = whisper.load_model("medium")
end result = model.transcribe("audio.wav")

As quickly as now we now have the captions, we divide them into groups of 20 sentences. We try this as we seen that if now we now have hundreds sentences the important thing phrases often should not acknowledged for every sentence. As quickly because the division is full we transfer it to the OpenAI API to generate associated key phrases for each sentence in a bunch. These key phrases symbolize the first ideas throughout the captions and are used to hunt out matching B-roll footage.

segments = end result["segments"]
extracted_data = [{'start': item['start'], 'end': merchandise['end'], 'textual content material': merchandise['text']} for merchandise in segments]
data = [x["text"] for x in extracted_data]
def split_array(arr, max_size=20):
# Document to retailer the minimize up arrays
end result = []
# Iterate over the array in chunks of dimension max_size
for i in differ(0, len(arr), max_size):
end result.append(arr[i:i + max_size])
return end result
# Occasion utilization
my_array = file(differ(100))  # Occasion array with 100 elements
split_arrays = split_array(data, max_size=20)

Now each of these group of 20 sentences is handed to a instant as could also be seen throughout the code beneath to find out seen key phrases which could be utilized to fetch associated pexels motion pictures. Lastly we merge the important thing phrases from each group to get the whole broll data which includes key phrases for each sentence of enter video.

from openai import OpenAI
import json
OPENAI_API_KEY = "openai-api-key"
broll_info = []
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY
shopper = OpenAI(
api_key=OPENAI_API_KEY,
)
for i, x in enumerate(split_arrays):
instant = """This generally is a transcript from a shorts video with 20 sublists. Each sublist represents a piece of the dialog. Your exercise is to find out a key phrase from each sublist that may be utilized to hunt for associated b-roll footage. B-roll footage ought to enrich the dialog issues and should be acknowledged such that it could give associated outcomes when searched in pexels api. Please current one key phrase per sublist. On no account skip any sublist and always give in order i.e from 0 to 19. Need output with key phrase and file index. Strictly give jsonnn**Enter**nn"""+str(x)+"""nn**Output format**nn[{"k": keyword1, "i":0},{"k":keyword2, "i":1}]"""
chat_completion = shopper.chat.completions.create(
messages=[
{
"role": "user",
"content": prompt,
}
],
model="gpt-4o",
)
broll_data = chat_completion.alternatives[0].message.content material materials
print("Data", broll_data)
try:
broll_data = json.plenty(broll_data)
apart from:
broll_data = broll_data.minimize up('```json')[1].minimize up('```')[0].trade('n', '')
broll_data = json.plenty(broll_data)
broll_data = [{"k":x["k"], "i":20*i+x["i"]} for x in broll_data]
broll_info.lengthen(broll_data)

Now that now we now have key phrase for each sentence of the dialog in a video, we’re going to select 50% of the important thing phrases for fetching B-roll motion pictures. This fashion 50% of the distinctive video shall be coated with b-rolls. Based mostly totally on requirement you could choose the subsequent or lower proportion of b-rolls.

import random
num_to_select = int(len(broll_info) * 0.5)
enumerated_list = file(enumerate(broll_info))
selected_with_indices = random.sample(enumerated_list, num_to_select)
selected_elements = [elem for index, elem in selected_with_indices]
selected_indices = [index for index, elem in selected_with_indices]
for x in selected_indices:
ingredient = broll_info[x]
extracted_data[x]["video"] = fetch_pexels_video(ingredient["k"])

With the important thing phrases in hand, we use the Pexels API to fetch associated motion pictures. First it’s important to create an account and Pexels and from documentation need to repeat the Pexels api key and paste throughout the beneath code. If we don’t uncover any motion pictures for a key phrase we return “Invalid key phrase”

import requests
PEXELS_API_KEY = "pexels-api-key"
def fetch_pexels_video(key phrase, orientation="panorama"):
url = f"https://api.pexels.com/motion pictures/search?query={key phrase}&orientation={orientation}&dimension=medium"
headers = {
"Authorization": PEXELS_API_KEY
}
response = requests.get(url, headers=headers)
data = response.json()
if data['total_results'] > 0:
video_info = data['videos'][0]
video_url = video_info['video_files'][0]['link']
thumbnail_url = video_info['image']
video_url = data['videos'][0]['video_files'][0]['link']
return {'video': video_url, 'thumbnail': thumbnail_url}
else:
return "Invalid key phrase"

Now that now we now have all the content material materials out there, we’re in a position to sew each little factor collectively. We iterate over all the sentences throughout the captions, creating clips for each sentence. If a sentence is chosen for a B-roll, we incorporate the corresponding B-roll video; in another case, we retain the distinctive clip. The B-roll clips are resized and trimmed to match the size of the respective sentences.

Now we add the audio once more to the created video above using concatenate_clips_with_audio function to sync every and generate the final word video.

import os
import requests
from moviepy.editor import VideoFileClip, concatenate_videoclips, concatenate_audioclips
from tempfile import TemporaryDirectory
from moviepy.video.fx.all import resize
def download_video(url, temp_dir):
local_filename = os.path.be part of(temp_dir, url.minimize up('/')[-1])
with requests.get(url, stream=True) as r:
r.raise_for_status()
with open(local_filename, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
return local_filename
def process_broll_clip(b_roll_clip, segment_duration, original_audio, start):
b_roll_duration = b_roll_clip.size
if b_roll_duration < segment_duration:
num_loops = int(segment_duration / b_roll_duration) + 1
b_roll_clip = concatenate_videoclips([b_roll_clip] * num_loops)
b_roll_clip = b_roll_clip.subclip(0, segment_duration)
else:
b_roll_clip = b_roll_clip.subclip(0, segment_duration)
b_roll_clip = resize(b_roll_clip, newsize=(original_clip.w, original_clip.h))
# Set audio from the distinctive video to the b-roll clip
b_roll_clip = b_roll_clip.set_audio(original_audio.subclip(start, start + segment_duration))
return b_roll_clip
def concatenate_clips_with_audio(clips):
audio_clips = [clip.audio for clip in clips if clip.audio is not None]
video_clips = [clip for clip in clips]
final_video = concatenate_videoclips(video_clips, approach="compose")
if audio_clips:
final_audio = concatenate_audioclips(audio_clips)
final_video = final_video.set_audio(final_audio)
return final_video
# Load the distinctive video
original_video_path = 'video.mp4'
original_video = VideoFileClip(original_video_path)
original_audio = original_video.audio
with TemporaryDirectory() as temp_dir:
final_clips = []
for part in extracted_data:
start = part['start']
end = part['end']
segment_duration = end - start
original_clip = original_video.subclip(start, end)
if 'video' in part and part["video"] != "Invalid key phrase":
print("Section", part)
b_roll_video_url = part['video']['video']
b_roll_video_path = download_video(b_roll_video_url, temp_dir)
b_roll_clip = VideoFileClip(b_roll_video_path)
b_roll_clip = process_broll_clip(b_roll_clip, segment_duration, original_audio, start)
final_clips.append(b_roll_clip)
else:
final_clips.append(original_clip)
final_video = concatenate_clips_with_audio(final_clips)
final_video.write_videofile('final_video_with_broll.mp4', audio_codec='aac')

Beneath is a demo video with AI b-roll added. Proper right here is the distinctive video https://github.com/Anil-matcha/AI-B-roll/blob/main/video.mp4