When I began editing video the medium’s primitive audio capability disappointed me. I had worked with eight or more tracks of film sound. Suddenly I had only two; and laying those tracks was a tedious business.
The shock was even greater when I moved from professional equipment to consumer gear in a video production classroom: A/B roll was hit or miss and post-syncing dialogue was painful. As for complex, multi-layered sound tracks, New Yorkers have the apt response: fuhgeddaboudit!
So I was stoked when I first encountered a nonlinear editing timeline, with its ability to wrangle dozens of audio tracks, just like film! I found two new features that even film audio lacked:
- A graphic waveform display that let me see sounds as well as hear them. I could now cut and sync them visually.
- A preview function that let me audition my work at full quality as I created it, instead of walking "deaf" into an expensive mixing session and hoping for the best.
At long last, video sound was playing with the grownups!
The digital advantages that I recognized were obvious ones; it took a while before I discovered two more fundamental advantages that lay below them.
Layered Means Better
To understand these advantages, look at a typical nonlinear editing timeline (not a storyboard style nonlinear editor) as displayed by Adobe Premiere 5.1 (Figure 1a). Fact is, this timeline is nothing more than a very long skinny matrix.
Re-size it to look more like a conventional grid and it might resemble Figure 1b. (For simplicity, we show only one video line.) The vertical axis displays four pairs of sound tracks: dialogue, effects, music and background. The horizontal axis lays out the program from start to finish (we see only a few seconds here).
The two fundamental advantages of nonlinear editing are expressed by those horizontal and vertical grid axes:
- Horizontally, you can select, place and manipulate individual sound components with single-frame accuracy.
- Vertically, you can build multi-layered sound tracks by working on one level at a time.
This complete horizontal and vertical control allows you to accumulate a complex, multi-layered sound track by successive approximation. Select, trim and place the dialogue, auditioning it repeatedly until you’re satisfied. Next, lay in sound effects one track at a time and as many tracks as you need. Check the results. Like it? Okay, then do the same thing with the music, then with the ambient sound, then with whatever other audio elements you need.
That, of course, is exactly what linear audio cannot do. Working with individual sound bites is a clumsy and difficult business at best. Though you can premix non-sync tracks, the process is tedious; and you still have to blend and lay your final audio in a single pass. If you don’t like the results you have to do it all over again. And again. And again. All too soon you find yourself giving up the struggle and restricting post-production audio to some music or narration (though not at the same time).
In digital sound, by contrast, you can shape its characteristics, sync it precisely to the picture and deposit it on one of several audio layers. Let’s take a quick look at each process.
Shaping sound elements means losing the bad parts, adjusting the volume and equalizing frequencies.
To demonstrate cutting out unwanted stuff, imagine that a director has marred your production track by talking right in the middle of a scene. To solve the problem:
- Cut out the offending comment.
- Find a piece of production track containing background sound only and duplicate it.
- Fill the hole in the dialogue track with the duplicate piece.
With a bit of practice, you’ll find you can perform a chore like this in about a minute.
You can also use the graphic volume indicator line to fine-tune sound levels. For example, automatic gain controls on most camcorders and VCRs are often fooled by silence into cranking up the volume excessively; so the first sounds are much too loud, until the circuit readjusts. With a visible volume control, you can add handles to the line as "hinges" and then drag the volume level up or down to create fades.
Finally, you can shape sounds by equalizing them with the controls in the audio editing software that’s usually supplied with sound cards.
The same technique of pushing pieces back and forth on the timeline makes it easy to synchronize dialogue and sound effects.
With dialogue, you may wish to continue the high-quality sound recorded with a closeup over a long shot, where the mike was much farther away. The trick in this case is to adjust picture to sound, rather than the other way around.
You can also double up sound that was recorded separately. In our example (Figure 1B), note that Her line overlaps His. We can do this easily with double dialogue tracks (and the long shot in a dark parking lot lets us get away with the brief visual mismatch.)
With sound effects, we’ve laid each CLIP or CLOP individually. This isn’t hard to do because the waveform lets us snip each footfall by sight, exactly at the head and tail of the sound. You can also do this with a Foley studio, creating the sounds to picture playback and then, fine-tuning the synchronization on the timeline (for more on Foley audio, see Real-Time Sound Effects: The Foley Way in Videomaker‘s July, 1999 issue).
With the ability to push around graphic representations of sound elements, you can easily time music to suit your program.
The most common practice is backtiming: laying a piece of music so that its big ta-daaa! finish coincides perfectly with an end point in the visual track. To backtime,
- Position the music so that its end matches the video’s end.
- At the desired in point, fade the already playing music up from 0.
- Clip off the unheard beginning portion of the music. (You don’t have to, but it reduces the file size of the music clip.)
Of course you can do just the opposite with music that runs too long: Start it at full volume, then, where you want to lose it, gradually fade it out.
A great trick is to use the two techniques together to create a crossfade:
- Lay in the first music selection from the head end.
- Lay in the second one from the tail end (backtiming).
- Fade down track one while simultaneously fading up track two. The result is a musical dissolve that can be very pleasing.
Or if you want a slight breather, end one music track with a fade out before fading in the second music track.
When you use these techniques together, they’ll give you control over your music that you never had in analogue editing.
We’ve already seen that the parking lot/club sequence in Figure 1B uses eight layers of sound. In fact, this is a very simple example. Many programs assign a separate dialogue track to however many people are in a sequence; and sound effect tracks can grow to as many tracks as you need.
Layering allows tricks like overlapping dialogue, and it also increases flexibility in effecting transitions. For an example, look at the transition from the parking lot to the club interior. Note that we’ve butted the music selections together at the video cut point so the deafening band music instantly and shockingly replaces the quiet outdoor music.
Now check out the background sound in Figure 1b. Notice that the club interior cross fades with the night sounds; and this audio dissolve is completed before the video cuts to the club. This is called a "split edit," with the audio leading.
Why not butt the two backgrounds together like the music selections? Because the growing din of the club underneath her line about noise gives her remark more weight.
Incidentally, note that movie sound doesn’t have to be consistent. In real life we would either hear the music and noise grow as we approached the club, or else it would hit us abruptly as we stepped inside. Either way, the music and background would transition together, rather than independently as they do in our example.
Sounds So Good
This is just a taste of what you can do with timeline-based sound cutting. By going digital you replace a primitive, limited, all-or-nothing audio editing method with a system so flexible, accurate, and above all incremental, that it would satisfy even Messrs. Dolby and Lucas.