Camera aiming using object tracking

Viewing 5 reply threads
  • Author
    • #81866

      Hey guys,

      I’m an amateur film-maker and robotics graduate student in Waterloo, Canada and I’ve recently been playing around with new and interesting tools for camera motion control. I’ve used the new DJI Ronin handheld stabilizer gimbal, built a camera dolly rig using an RC car as well as used a drone to film a couple of sweeping shots of an object in a field.

      I quickly realized that it’s actually pretty difficult to control the camera while simultaneously running around with a handheld unit, driving the motorized dolly or flying the drone in the air. Anyway, so I’m a roboticist and hacked together this working prototype that uses the video stream to track anything I want, using computer vision algorithms.

      Here’s a quick youtube video to show what I mean:

      So far, I programmed in smooth-motion and the ability to track and frame anything I select in a box. In the future I am going to make an android/ipad app so that you can just multi-touch define a subject with 2 fingers.
      After looking into the control interfaces for most of these gimbals and motorized tripod heads I’m pretty sure I can make them follow my commands, in addition to commanding the motion paths of the drone or rc car itself to follow something (ie. a bike, animal, car etc.)

      I was just wondering if anyone out there would also interested in this tech? It’s easy enough for me to make and share, and any feedback on cool ideas to implement would be great


    • #211053

      Incredible.  This could be to the hand held gimbal what the hand held gimbal was to the steadycam.  A game changer.  If this could also be setup as an aide to the gimbal operator that would be pretty slick as well. 

    • #211077

      You need a post a video of the camera operating as you do the movement.


      As far as applications, there are some out there already that basically control a pan and tilt tripod head that are designed for sports activities.  They use a transmitter of some sort and sell in the $300 range if memory serves called the soloshot.  It has taken off in some of the active sports realms but by definition are incapable of doing the close work your demo provided. (You've got to be something like 20-30 ft. away for the dongle-tracking system they employ to work.)



    • #211116

      I have been looking for just such an application.

      One that can operate my Ronin Gimbal and keep the camera focused on a fixed point in the distance.

      If you program an app that would work through my ipad I'd be a buyer, or a contributor for a Kickstarter campaign.

      I know several DP's who would welcome such an app.

    • #211286
      Just found your write up,
      awesome work!!
      im very interested in this kind of tracking. and have no idea how to go about it.
      Im a sports action cameraman, specializing in following the athlete.
      I own 2 movi gimbles, I have used both single use mode and have a friend/partner who remote controls the gimble.
      would love to find out if your idea would work with a movi.
      Let me know if you ever want to try this out in the filed would love to collaborate with you if your interested
      hear are a couple projects I worked on from last year.
      Kirk Bereska
    • #211363


      I am interested in a similar, yet different thing: a motorized tablet-(or laptop)-controlled tripod head which can track an object and adjust its position in order to keep it in frame. The tripod head will also be able to capture the metadata of its position. This is so that the tripod head can replicate the same shots over and over again. It will also have a gyroscope, so that it can adjust for tilt (for example if it is on a tilted surface). However, gyroscope adjustment should be optional when playing back motions, because there are cases when you might not want to adjust for tilt. The cooler thing, however, is that you can export your metadata as 3D camera tracking data, for use in compositing software. While this will only work for tripod shots where the tripod itself is still (not on a dolly), that is also its greatest strength. While software like mocha or syntheyes is great for handheld shots, it is not that good on tripod shots, and utterly useless when the shot is entirely still (neither the tripod nor the tripod head moves). Also, it will be able to import 3d camera data, and replicate the motions of virtual cameras. While this is only useful when the virtual camera behaves like it is on a tripod, it is still extremely helpfull. Imagine that you have purchased a ready-made bluescreen clip of a t-rex, for use in a project. The virtual camera in the bluescreen clip does not move, but it is at a certain angle. You do not know what angle this is, and therefore do not know how to angle your real-world camera. This is a tedious guess-and-test process of filming, transferring to a laptop, keying, rendering, and adjusting the camera again. It can take an hour or more to film a single shot using this unscientific method, and even then the end result is often innacurate. While this is not a problem for hollywood studios, who have their effects custom-made after they film, independent and low-budget filmmakers do not have this luxury, and they rely on purchasing premade assets. This tool, if made, could revolutionize the independent film industry.


      Please contact me or reply to this comment (I really want to hear what you have to say), and please make a kickstarter/indigogo project out of this idea. I will be your first backer. 



Viewing 5 reply threads
  • The forum ‘Production Gear’ is closed to new topics and replies.


Panasonic LUMIX GH5s w/ Leica lens

Given away to one lucky winner

Competition is open worldwide