Overview
In-Camera VFX (ICVFX) refers not to a single fixture or grip device, but to a Virtual Production shooting technique: the final image background is created directly in-camera during the shoot, rather than in post-production compositing. In its currently widespread form, large-scale LED walls (LED Volumes) are used, which display a digital environment in real-time, in front of which actors and real set constructions are filmed.
In contrast to the classic greenscreen/chroma key, the crew sees the finished image frame immediately in the viewfinder. This eliminates the need for post-production keying and background insertion; directors, camera operators, and actors work against a visible image instead of a green surface.
How It Works
Three components interact:
- Real-time Render Engine: A game engine – commonly Unreal Engine by Epic Games – calculates the digital scene in real-time and outputs it to the LED wall.
- Camera Tracking: The position, orientation, and movement of the real camera are continuously captured and transmitted to the engine, so that the virtual camera moves in sync and the perspective parallax of the background is correct.
- Frustum Rendering: Within the camera's field of view (Inner Frustum), the image is rendered in high resolution and with correct perspective; the surrounding area (Outer Frustum) primarily serves as a light source and reflection surface.
Significance for Lighting and Reflections
For lighting and set work, the LED wall itself is an active, large-scale light source. It casts the light of the displayed scene – color, brightness, direction – directly onto actors and the set. This creates contextually appropriate interactive light as well as realistic reflections on eyes, skin, paint, glass, and metal, which would have to be laboriously added in post-production in a greenscreen workflow. If the background changes (e.g., sunset, passing lights), the light on the faces changes automatically with it.
Usage and History
The LED Volume variant became widely known with the series The Mandalorian (starting 2019) and the StageCraft system developed by Industrial Light & Magic (ILM), which combined Unreal Engine with a semicircular LED wall and ceiling. Since then, ICVFX has become established in series and film production. It should be noted that the term, in a broader sense, also encompasses practical effects created "in-camera" without post-production (e.g., multiple exposures or real optical tricks); however, in current usage, ICVFX most often refers to the LED Volume/real-time rendering technique.
