Overview
3D Tracking (also known as 3D Camera Tracking, Matchmove, Matchmoving, or Camera Solving) is a visual effects (VFX) technique where the movement of a real camera is analyzed from captured live-action footage and reconstructed in the three-dimensional space of a software. The goal is to integrate computer-generated elements (CGI), set extensions, or replaced backgrounds into the live-action shot so that they precisely match the original camera in position, scale, orientation, and movement.
In contrast to 2D Tracking, which only tracks movements in the image plane (X/Y), 3D Tracking also considers depth (Z-axis) and thus perspective, camera rotation, and focal length. 2D Tracking often provides the initial data from which the three-dimensional camera solve is then calculated.
Principle of Operation
The software tracks distinctive image features (feature or tracking points) across the image sequence. From their displacement from frame to frame, the camera's movement and orientation over time can be triangulated. The result typically includes:
- a reconstructed camera path (position and rotation per frame),
- a point cloud, which describes the location of objects and surfaces in space,
- an estimation of focal length and lens distortion.
This solved virtual camera is then handed over to compositing and 3D programs so that rendered elements appear from the exact same viewpoint as the live-action shot.
On-Set Considerations
Although the actual solve takes place in post-production, the quality of 3D tracking is largely prepared on set. Common practices include attaching tracking markers (e.g., adhesive dots or crosses, especially on solid-colored surfaces like greenscreens), noting lens and focal length data, and capturing reference measurements and, if necessary, survey data of the set. Clearly visible, static structures and sufficient parallax facilitate the later solve; pure pans without positional change make it more difficult.
Software
Tools used for 3D Tracking in film and TV include, among others:
| Software | Manufacturer / Provider |
|---|
| 3DEqualizer | Science.D.Visions |
| PFTrack | The Pixel Farm |
| SynthEyes | Andersson Technologies |
| Mocha Pro | Boris FX |
| CameraTracker (in Nuke/NukeX) | Foundry |
| Camera Tracking (integrated) | Blender Foundation |