Tutorial  Rotoscoping: fix goofs, change virtual sets, or make your hero fly

Interactive Tutorial Content

Click here to view the tutorial video for Rotoscoping.

How to fix on-set goofs, change virtual sets, or make your action heroes fly through the air amidst gunfire, snowflakes, or magic dust.

Rotoscoping is one of my favorite things in Visual FX. I know that sounds crazy, but the results of rotoscoping can be so rewarding. Doing the actual work can be tedious, but, when it’s all said and done, you can sit back and be proud of your accomplishments.

What is rotoscoping? Here’s the simple version: rotoscoping is articulating a mask to either add something to or remove something from the frame. You do this process frame by frame, and that’s why it’s so tedious and time-consuming.


How to Make a

DIY Green Screen

Free eBook


How to Make a

DIY Green Screen


Thanks! We will email your free eBook.

This article is not just a step-by-step guide. It’s actually more about sparking ideas on how you can use rotoscoping and masking in order to make your films better by increasing your production value. There will be two examples provided. If you would like to see more, visit the video tutorial section at www.videomaker.com, where we aren’t limited by word count and page space.

We used Adobe’s After Effects, your software tips may vary. Remember, rotoscoping can cover a broad range of effects. I hope to show you some cool ones in this article.

Sample 1: Object Removal

Have you ever had a shot ruined because the mic dipped into frame? What if you can’t get the mic close enough to get good sound?

Well, think about this for a second. Maybe you can lock the shot off (no tilting, panning, zooming, dollying, etc.) and get the mic closer by actually sticking it in the frame. I don’t recommend doing this for every shot, but for special circumstances, it can help.

Notice the mic in the frame of the photo.

Step 1: Shoot your scene.

If you must shoot your scene with the mic in the frame, take care to not overlap the mic and the actor. You also want to make sure you don’t move the camera or zoom at all. Keep the lighting the same as well. This is crucial to matching the cutout section with the rest of the footage.

Step 2: Physically remove the mic from the shot, revealing the wall behind it.

Notice the mic is no longer in the shot, but nothing else has changed. The lighting, camera position, lens, etc., have all stayed the same. Removing the mic reveals the wall behind it. You’ll use that section to fill in the blank area that we will cut out. This type of footage is usually called the clean plate, and that’s what we’ll call this footage from here on. Although I didn’t do it in this example, it is best to move the actor out of the frame too.

Step 3: Digitally cut out the mic.

Using the Pen Tool in After Effects, cut out the section of the frame where the mic can be seen. Depending on the movement of the mic, you may have to animate or change the position of the mask. (You animate the mask by pressing MM on the keyboard to bring up the mask properties. Set a keyframe for the Mask Path, then move to the next frame, and change the position of the mask or mask points. This will automatically set another keyframe under the Mask Path.)

Depending on your shot, you may have to feather the mask a few pixels. Make sure your mask is also set to Subtract. This will subtract the portion of the image you just cut out. The default is Add, which means it keeps the portion you selected and removes the rest of the image – so make sure you select Subtract.

Notice the section of the frame is now missing! Don’t fret!

Step 4: Fill in the blank.

In your timeline, drop the Clean Plate (footage without the mic physically in the shot) below the footage with the mask (the footage with that big gaping hole where the mic used to be).

Hey look! The mic is gone! Using this technique, you were able to get the mic close, thus maintaining good sound quality for the dialogue, without having to worry about keeping the mic out of the shot! Remember this while shooting, because the number one thing that people notice in a film is bad sound. Bad picture can oftentimes be excused, but that’s not the case with sound. Keep your production value high, and your movie will benefit!

Sample 2: Explosions

Yeah, you know you were waiting for this one… As we’ve mentioned before, you can go to www.detonationfilms.com for stock footage of explosions!

Step 1: Place in the explosion.

You’ll notice that part of the explosion is covering our actor (F).

Step 2: Draw the mask.

Here’s where the roto work comes in. We place our explosion layer on top of our live action footage shot, so naturally, it’s going to be above (or covering) everything in the shot. In this particular scene, we want the explosion to happen behind the actor, so we have to do some roto work.

Simply select the explosion layer, and cut a mask around the actor. You don’t have to worry about the side that the explosion isn’t reaching (the left side of the actor); you just want the actor’s right side to cover the explosion. Select your Pen tool and draw a mask around the actor where the explosion needs to pass behind him. Make sure the mask is set to Subtract, so it subtracts the section you select.

Here’s a shot of the mask I drew (G).

Here’s a shot of the screen without the mask lines showing (h). You can see that the actor now covers that part of the explosion, creating the illusion that it’s behind him. We added a 2-pixel feather to the mask in order to make the edges blend better. Edges are not sharp in real life, so play with the feathering to see what works best. You can find the Mask Feather options by pressing MM on the keyboard with the layer selected.

The only problem is that this mask is effective for only one frame. If you move to the next frame, the mask won’t follow. You have to animate it to follow, or outline, the actor.

In this picture, I’ve moved 5 frames forward (I). You can see the results of not animating the mask. Part of the explosion is now covering our actor’s head, and there is a gap between the actor and the explosion. Not good…

Step 3: Animate the mask.

So here’s the same shot but with the mask animated to follow the actor. (When I say “animate the mask,” I mean moving the points of the mask so that the mask continually outlines our actor. You’ll have to keyframe the Mask Path in order for After Effects to remember the different positions of the mask. See the Rotoscoping Tutorial video for more information.)

Here, you’ll notice that we have animated the mask to follow the outline of our actor (J). Notice the gap and the explosion covering his head are gone, and everything is as it should be.

Just animate the mask for the entire shot, and you will now have an explosion in the frame that stays behind the actor, giving the shot realism through correct depth.

It’s Worth It!

I hope I didn’t confuse you too much. This can be very involved and complicated material, but – trust me – it’s well worth learning. You’ll be able to do so much more, both technically and creatively! Watch our video to see what I mean.

As my favorite director always says, “If you’re both technical as well as creative, you’re unstoppable!” How do you think we shot that video in one day for no money? Until next time!

Interactive Tutorial Content

Click here to view the tutorial video for Rotoscoping.

The Videomaker Editors are dedicated to bringing you the information you need to produce and share better video.