I find crowd chatter annoying and try to keep it out of the audio mix (or extremeley low, under the background music) unless I'm recording dance scenes, a toast, a dinner blessing, or a personal greeting.
Another thing to consider is that stereo ambient sound adds to the realism of the audio-visual experience. People enjoy hearing laughter and bits of conversation (in moderation, of course) all around them, so other than for personal greetings -- when I give the person a mono handheld mic -- I use an external stereo mic to record "regular" sound. I don't bother using a shotgun mic because the P.A. system and room reflection sound fill most of the stereo spectrum. Take advantage of those two (or more) audio channels. The VX2000 has a built-in stereo mic, and I'd feel comfortable using it for "regular" sound where a little stereo ambience is appropriate. Besides, you can always EQ your audio tracks in post to compensate for mic characteristics (too bassy, too tinny, etc.).
If, on the other hand, there's no P.A. system, I'd clip a lav mic on each scheduled speaker just prior to their speaking to make sure I could record their voices over the din.