I have used a small Olympu

I have used a small Olympus recorder in the past – It records at 22.1 khz and that caused all sorts of problems with drift. Try recording a longer (20 minutes) piece on the recorder and your camera and see if you get the same issues I did (I needed to cut up and re-sync every 20 seconds or so).