How to Use Rotoscoping to Create Video Special Effects

What’s up, everybody? Paul Del Vecchio here, Triple E Productions, Video Maker Magazine. And we’re going to be doing some rotoscoping techniques today.

Let’s drag in our explosion. We’ll put that on top. And let’s just move this. Let’s trim this a little bit and let’s see, all right. So we want the explosion to happen appearing in the background so let’s rotate it so it matches the plane. So it matches the grass over here. Let’s move that up around there. That looks good. It lines up with the plane here. Okay, so now what we want to do is we want to take this and change the transfer mode here to screen. If you don’t see this right here the mode you can just hit F4 and just change it to screen that will get rid of the black background.

Now we want to offset this explosion so that it starts like maybe around there it looks good. And it starts around there. Now you see the explosion appears over Dave and we want to look as if it’s appearing behind Dave. The reason why it appears over Dave is because it’s a layer of footage that’s over our bottom layer, which is just Dave running. So we need to make the – we need to give the illusion that the explosion is behind Dave. So basically let’s reposition this because this looks like it’s in front of him so let’s just grab this and put it in the background over there. It looks good. Let me just check that out. Yeah, so that looks like it’s behind Dave and if we just hit page down and just move through it frame by frame we can kind of see that, you know the explosions happening but then all of a sudden here it’s overlapping with Dave. And you can see that. It’s kind of covering him up. If I turn off the explosion layer you’ll see that it’s kind of over his arm. So now the perspective is all screwed up and what we want to do is make it appear as if the explosion is happening behind Dave.

Now you can see that it’s now overlapping and that overlapping and you know that’s no good. We want to make it look a little bit more realistic here. So what we’re going to do is do a little technique called rotoscoping. And the first step would be to duplicate our footage of the actor running. And then we want to place that on top of the explosion. Now the explosion disappears because obviously that layer is covering it up. So just hit T and bring the opacity down to like 50 percent, 30 percent whatever works best for you. And when we bring it down we can kind of see the explosion happening underneath. Now remember this is a little confusing so just remember that this top layer is opaque. And if we turn off the bottom layer, which is the same it’s a duplicate of the footage you can kind of see that you know that’s what we have underneath this top layer. If we turn the opacity back up of the top layer all that disappears. Now remember this bottom layer is the same footage as this top layer here but it’s turned off right now just so that you can kind of see as I turn this opacity down this is what beneath it.

Now if I turn this bottom layer back on you see everything come back. And that’s because it’s the same exact footage as the top layer. The only reason why the explosion is seen is because if you take a look at like the hierarchy here it’s like this top layer is semi-transparent because it’s at 37 percent opacity. So beneath that we’re going to see the same footage and on top of that we’re going to see the blast. So that’s why the blast looks kind of transparent and the rest of the footage doesn’t. So the footage, the blast layer is just kind of sandwiched in between the other two layers. So what we want to do, the reason why we turn the opacity down is so that we can see the blast and we can see at what frame it starts to overlap our actor, which is this frame right here.

So what we want to do now is kind of zoom in here, take our pen tool, and start a mask. And just kind of cut around where we think our actors arm is because you want to outline the actor. So now with the top layer selected remember we want to place this footage of Dave on top of the explosion. So basically you think about it; it’s going to be – let’s turn these layers off – it’s going to be our bottom layer, which is basically everything there. And then our top – and then our explosion layer, which is on top over here and that’s going to appear, you know it’s going to cover up Dave here. So in order to get Dave to appear in front of the explosion here we’re going to have to put another layer of Dave on top of this explosion, which is this layer, right here. That’s the layer of Dave on top.

So let’s do Dave top. Just name these. This is our blast. And this is Dave bottom. Okay. So now with the top layer selected we want to take our mask and just kind of outline Dave. You know outline where this fire explosion layer is sort of covering him up. We can just you know go a little, put a few points in here where we think that Dave is, you know Dave’s arm is and then we can just kind of go around here and outline Dave with this mask here.

Now the explosion isn’t going to go any further than around here so we don’t really need to worry about anything beyond this point right here. So we’re just gonna round this off a little bit and this will save us time. We don’t need to worry about this section here so we’ll just do a shortcut and close off the mask there.

Now if you look at this – bring up the mask properties. And now we’ll see that – let’s bring our opacity of the top layer back all the way up – you’ll see that where we put the mask everything inside of this mask is being kept because remember this is the top layer so it’s keeping everything inside of the mask. Everything outside of the mask is being subtracted or cut out. Now if we solo the layer you’ll see that. So this whole section is kept and over here where the explosion is is being cut out. So that, you know, this part of the video doesn’t overlap the explosion and then we can’t see it. So we need to cut out this section, we’ve outlined Dave, and then once we unsolo this you’ll see that the explosion fills in all that and the bottom layer also fills in all of this part of the frame that we cut out.

Now if you look here you’ll see that the edge is kind of sharp and that doesn’t really blend too well. So we can just bring up our mask properties and just do a 1.5 pixel feather. It’s going to be different for you but you’ll see that – it really depends on your footage – but you’ll see here that if we remove the mask here you’ll see that it blends a little bit better. Because when you have objects in front of each other in the real world the edges of the object in front of the other one they’re not really sharp because they’re actually soft. So that’s why we added the masks so that it blends a little bit better and doesn’t look too artificial.

So let’s bring this mask back up and it’s a little hard to see so if we just double click on this icon here or single click on it we can change the color. Let’s make it green so we can see it. All right. Now we can just adjust these points here on the mask to kind of see where Dave’s arm is. And it’s looking like – let’s see here – so his arm is over here, so we want to just outline his arm. And you just want to make sure that you’re not covering up his arm at all. So I just place that in there, feathering helps to blend it off a little bit better.

Okay, so now that we have done that for one frame, now remember, this is not going to move. The mask is not going to move with the actor we have to do that part. That’s the hard part and that’s the tedious part but we have to do it. And that’s what rotoscoping is just basically animating a mask. So you’ll see that the mask doesn’t move with the actor and we need to make it move with the actor. So in order to do that we have to click right here on mask path. We want to click this stopwatch so that we can set a key frame. And then from there we can hit page down to go to the next frame. And we can grab a little point here and just kind of move our mask around to see where we need to place it so that the explosion isn’t overlapping. And somewhere around there is good. And now what we want to do is move these points individually so that we can get them the , you know just fine tune it so that the explosion isn’t being cut off where it’s not supposed to be cut off. So around there looks good. We want to move this in here closer to his arm and over here closer to his head. And all these other points seem okay because the explosion is not anywhere here by his head yet.

So that looks good for the second frame. Let’s move this over here just to clean it up. But yeah, that looks good for the second frame. You might want to hit this little button over here to toggle the mask off so you can kind of just take a look at it. It looks good. Turn that back on and hit page down to move to the next frame and adjust some of these points a little bit more. You want to move it so that we can see where his arm is so around here. Let’s move this one down here. And this closer to his arm over here. Change these handles so that they’re not so wild. And let’s move this out and see where, so around here. And take this point and drag it out. Let’s change this hand over here and move this one to approximately there. Let me just clean this up a bit around his head. All right, so zoom out a little bit and check how that looks. It looks pretty good.

Page down, move onto the next frame and you just want on keep on doing this for the duration of the shot. So let’s see what’s going on over here. All right, so his arm seems to be extended up around there. It’s a little hard to see because it blends in with the brown trees in the background. He has a brown jacket on but that looks pretty good. Over here you’ll see the explosion is over his head so you just want to take a point and move it out so it outlines his head a little bit better. You might have to add more points too. Over here I think we’re going to need to do that. So let’s just take our pen tool, add a point, click, and drag it out. And while you’re doing this while you have the pen tool active, if you hold alt it’s a little selection tool so you can draw a box around these little points and it will just select the each individual point instead of selecting a line or the entire mask. And also if you hit control you can kind of select the point and move it around.

So let’s just clean this up a little bit and move that over. And take this move it out. Zoom over here you’ll see flames over his head over here so you just want to take that point and just move it out, and same thing with this one. Now this one’s a little strange with the handles so let’s just straighten those out a little bit. Let’s see how this looks. Yeah, so that’s good. This point is good. And just clean this up a tad.

All right, so we’ve done four frames let’s do one more just so you guys can sort of see what it looks like. Let’s move this out a little bit. Yeah, so you start to see you know the arm is like extending over here so you just want to kind of move this out just to see where the arm is. There you go. So you can see this, you just want to grab this and just kind of outline the actor as you’re doing this. Try to make it a little neat. I’m going to put another point here just to have one for the tip of the elbow. Straighten out these handles here. Move this and move this along his body over here just so that we’re not missing any of the flame shooting off the explosion there. And let’s see what’s going on by the head over here. All right. And this you see is kind of overlapping the head so you want to move this out a little bit and drop it down right there. This we can throw over here.

So we got five frames now that we’re working with and that should be enough for you guys to get the idea. All right, so let’s just play this back now. Well, as I scrub through here you’ll see that the mask is now moving with the actor. So that’s a good thing. That’s what we want to do. So let’s just scrub through this and or actually let’s do a RAM preview instead and check out how this looks. So you’ll see that explosion is now not covering the actor and it’s visible where it should be, where the actor is not covering it up. So that’s good.

Now let’s turn the mask off and you just want to make sure that you watch it frame by frame just to make sure there’s nothing strange happening that you need to adjusts. Anything that looks artificial or anything that looks just a little strange or off, but yeah, basically that’s a – it’s a very useful technique. It’s a very time consuming technique but it works pretty well.

Another thing here what we want to do is we can use this technique for something other than putting in an explosion or something like that. We can actually use it for object removal. And I’m just going to show you guys this thing real quick. We got Dawn over here. So let’s, let me show you this real quick. Okay, so here’s our actress, Dawn, and you can see over here that the mic is in the shot over here. If I just take a curves adjustment, drop it onto the shot, and brighten it up just a little bit so that we can see what’s going on. Now you can see the mic in the shot clearly.

So right here you’ll see me, right, right here you’ll see me walk into frame and remove the mic. All right, so we got Dawn right here and we’re just going to trim this so that, so that we can just work with what we need to work with. Okay, so now we got this little section here and you’ll see right here that the mic is in the shot and then all of a sudden, you know, you see me walk into the frame and take the mic out of the shot.

So basically that’s what you want to do. When you’re shooting you have to keep this in mind too. This is what you need – this is what you want to do. Over here you’ll see that the mic is obscuring the wall. It’s blocking the wall over here but the camera, you know; take note of this that the camera does not move. So basically between you know, as we travel further down the timeline we’ll notice that the mic is not there and the camera hasn’t moved so it just reveals the wall behind the mic.

Now since the camera hasn’t moved we can use this section of the video just in this corner to replace the section of the video where the mic is. So first step here is just duplicate the shot. That was the effect. Duplicate the shot, select a layer, duplicate it, and then what we want to do is just take this section of the footage where the mic is not in the shot and we want to trim this. And now we want to move this to where we want to remove the microphone, which over here we’ll just say is the beginning of the shot here. So now what we can do here – now, okay, take a look at this first. You’ll see that this is the shot where the mic is in the frame.

So if we just shut off this layer for a sec, the top layer, we’ll shut it off, and then you’ll see that the bottom layer is where I took out the mic. But you’ll see that the wall doesn’t move because that the camera hasn’t moved. So we can use this bottom layer here as a clean plate or a clean wall section. What I mean by that is mic. So inside this circular little section that’s where we’re going to subtract from the image. So select a top layer, hit M M to bring up the mask properties, the mask is what we just drew, and then change this to subtract. So now it subtracts whatever’s inside of the mask right here. So now if we just scrub through it you’ll see.

Now if I turn this mask off you’ll see that the mic appears. Now if I turn the mask and change it to subtract and subtract everything that’s in the mask it disappears. And just to give you a better idea I’m going to solo the top layer so you can see what’s happening. So the mask has cut out this section of the video, you can see the transparency graded underneath. So now when we make the bottom layer visible just that section of the bottom layer fills it in. So now we’ve successfully removed the mic from the shot.

Now again, you want to keep this in mind when you’re shooting if you have a good take and the camera’s stationary, locked off, it’s not moving, you can take a second shot, you can take a second shot of the same scene. So you want to keep this in mind when you’re shooting. If the mic accidentally dips into the frame you can remove it. And then, you know as long the camera is not moving because then we’d have to go into like motion tracking and stuff like that. But if the camera is not moving it’s a stationary locked off shot then you know the wall doesn’t move in the background obviously. The actor, as long the actor doesn’t overlap with that section.

So it’s a good little technique if you have like a good take and you know, somehow the mic dipped into the shot. And you still want to use that take and it doesn’t, the mic doesn’t overlap with anything that’s moving in the frame, you can remove it if you just take the mic out of the shot and then use the background behind the mic as a clean plate as we did here.

Okay. So this is just another technique. It’s not really; I wouldn’t really consider this rotoscoping as much as garbage masking, but it would be rotoscoping if you had to animate the mask if the mic was moving.

Here’s another good example. This is a little bit more advanced. I’ll just open our Netflix spec commercial and you’ll see that there are a bunch of layers here, but the layers that we have to worry about are only three, which are the raw footage, the explosion, and then the rotoscope layer with just the actor that was extracted from the shot using rotoscoping. And once this opens up you guys will see that.

Okay. So if you take a look at this shot you’ll see that there’s a lot going on here. Let me just move into this precomp right here where we have all our footage. These layers you can kind of ignore a lot of them. You only have to worry about three of them because a lot of them are just tests layers and things like that that I never removed. But if you – let me just take out some of these. These are no longer available and they’re not really necessarily anyway.

Okay. So you just want to worry about three layers. One is the bottom layer, which is our regular footage, which is this one right here. I’m going to solo it so you guys can see what’s going on. So it’s just a footage of our actor running pretty much. That’s all it is. Just footage of the actor running. Okay. So move down one, all right. On top of this layer now we have our fire or explosion layer. And you’ll see that the explosions are laid in place but it looks a little weird because they’re on top of the actor. And then on top of this layer we have our layer where we applied rotoscoping, so that’s just the actor. And if you want to just check out this layer with you know the top layer, the rotoscope layer it’s our actor. I’ll turn on the transparency grid so you can see that. And then if I – if you view the masks you can kind of see what we did here. Yeah. So here we go you can see the mask and this is just the same footage as our bottom layer but it’s rotoscoped so that the actor was extracted from the shot. So then if I put in the fire layer you’ll see that the actor is on top of, the actor is on top of the explosions. And then bring in the background layer you’ll see that that fills in the background.

Now I should do this from the bottom to the top just so you guys get a better idea of what’s really going on here. So, all right, so this is our bottom layer. Again, this is just an actor running; nothing is behind him just him running. Then we placed in the explosions. But the problem is that the explosions are all in the right spot the only thing is that they’re covering up our actor and therefore it just ruins the illusion that they’re supposed to be behind the actor. So now we took that rotoscope layer of just the actor and placed that on top of everything. And now that gets placed over the explosions. So now the illusion has been created where the fire is or the explosions are behind the actor.

Now if I selected, say you’ll notice I’m just going to solo it just so you guys aren’t distracted by all the other layers here. But you’ll notice here that there are a bunch of masks around the actor. And they all move and they’re all animated to match the actor’s movement. Now the reason why we use a bunch of different masks instead of one large mask is just so that you don’t go insane. I mean imagine trying to animate one mask that – I mean look at the way the arms move here. They swing up and they bend and they swing back down. They overlap with the body and so they’re just changing position like crazy. So what we wanted to do here is this the better way of doing it because if you try it to do it with one mask you have to worry about all these tiny little points and animating all these tiny points. And it just becomes like a really complex shape. So you don’t really want to deal with that. I mean look how simple this shape is for the arm, not too many points and it’s a very simple shape. I mean the same thing for the head. But just imagine if you had one mask for all of these body parts. It would be like, you know, a little lump here for the head. A little long kind of rectangular almost section here for the arm. Then you got to go down along the side of the body to the leg and it just gets all crazy. So basically what you want to do is you want to make sure that for every body part or extremity of the body you want to use a mask for each one individually. So one mask for one arm, one mask for the other arm, you know each leg gets their own mask. The body gets its own mask and the head gets its own mask. It just makes things a lot easier. It makes them easier to animate. It make rotoscoping – because rotoscoping is a very like tedious process so it just makes it a lot easier. It makes a little neater. And you can kind of keep track of your masks a little bit better. You know you can kind of just go in and name your masks. Like you know you click on this one and you can name it like body and the next one you can name arm or head. So it just makes things a little bit more organized. So keep that in mind when you’re doing this type of work.

So that’s a brief overview of rotoscoping. I showed you the technique in the first section and then we went on to do object removal and also more complex rotoscoping techniques for extracting somebody from a shot and placing them back on top of other effects elements in order to get perspective right and things like that. So rotoscoping is a very powerful technique. It can actually save you a lot of time on set and it can also help you be more creative by allowing you to think of you know, where before you would be like oh, well I could never add explosions. How would I do that? You know learn rotoscoping, learn a little bit of compositing and you can have cool explosions in your scenes and that will up the production value of your project.

So all these like basic visual effects here are good to know so that you can come up with these things on set and then save time on set. And just come up with ideas on how to solve certain problems. So this is just a brief overview and this something that I think that all you guys should try to learn because it will definitely improve your filmmaking.

Once again, my name is Paul Del Vecchio for Triple E Productions and Video Maker Magazine. I hop0e this tutorial will help you guys out and good luck with everything. And you can find me on the web at www.triple-e-productions.net or you can find me on my blog at www.pauldvblog.com. Thanks guys, once again this is Paul Del Vecchio for Video Maker Magazine and Triple E Productions and good luck with all your projects. Thanks.

[End of Audio]