Detecting Deepfakes Using Lighting Physics
Light obeys physics. AI-generated video often does not. Shadow directions, reflections, and facial lighting provide powerful forensic signals.
Detecting Deepfakes Using Lighting Physics
Light behaves according to well-understood physical laws — it travels in straight lines, reflects predictably off surfaces, and casts shadows whose geometry is determined by the positions and sizes of light sources. When an AI generates or manipulates video, it must reproduce all of these behaviours simultaneously and consistently across every frame. Even small failures can leave forensic evidence that trained analysts — and automated systems — can detect.
Why Lighting Matters for Detection
In authentic footage, every pixel's brightness and colour is the result of a single, physically consistent lighting environment. Shadows, highlights, reflections, and diffuse shading all derive from the same set of light sources interacting with the same set of surfaces. A generative model must satisfy all of these constraints simultaneously — and across time — to produce lighting that withstands forensic scrutiny.
Current AI video generators learn lighting distributions from training data rather than performing ray-tracing or radiosity simulation. This means they can produce statistically plausible lighting but may fail to maintain strict physical consistency, especially in complex multi-source or dynamic lighting environments.
Shadow Inconsistencies
Shadows are among the most informative forensic signals because their geometry is tightly coupled to the 3D scene layout. Key shadow properties that forensic analysts examine include:
- Direction: All shadows cast by a single light source should converge toward (or diverge from) a consistent vanishing point. AI-generated scenes sometimes contain shadows pointing in incompatible directions.
- Hardness: Shadow edge sharpness depends on the angular size of the light source relative to the object. A small point source creates hard-edged shadows; a large diffuse source creates soft ones. Inconsistent shadow hardness within a single frame may indicate synthesis.
- Multiple light sources: Real scenes with multiple lights produce multiple shadow sets with predictable overlaps. AI generators sometimes produce shadow configurations that cannot be explained by any physically realisable light arrangement.
- Shadow-object correspondence: Every opaque object should cast a shadow consistent with its shape and position. Missing shadows, extra shadows, or shadows that do not match their casting object are potential indicators of manipulation.
Reflections as Forensic Evidence
Reflections obey the same physical laws as shadows and provide independent verification of the lighting environment:
- Specular highlights: The position of a specular highlight on a curved surface encodes the direction to the light source. Inconsistent highlight positions across objects in the same scene suggest the lighting was not physically unified.
- Eye reflections: In close-up footage of faces, the corneal reflection (catchlight) acts as a tiny mirror of the environment. In authentic video, both eyes reflect the same scene. AI-generated faces sometimes produce catchlights that differ between left and right eyes or shift inconsistently across frames.
- Surface reflections: Glossy floors, windows, and metallic objects should reflect the scene in a geometrically consistent manner. Reflections that show content not present in the scene — or that fail to update correctly as the camera moves — can indicate synthesis.
Facial Lighting Mismatch
Face-swap and face-generation models often struggle to match the illumination of the target scene. A 3D face is a complex surface with strong self-shadowing (nose shadow, under-chin shadow, orbital shadows), and the shading pattern changes dramatically with even small shifts in light direction.
Forensic analysts can estimate a face's apparent illumination direction using spherical-harmonic decomposition and compare it to the illumination implied by the rest of the scene. A statistically significant mismatch suggests the face may have been composited or generated independently.
Real-World Examples of Lighting Failures in AI Video
While specific examples evolve as generators improve, common failure modes observed in recent AI-generated video include:
- A subject walking outdoors where the facial shading suggests overhead noon light, but the ground shadows indicate a low-angle sun — a physically impossible combination.
- Interior scenes where specular highlights on a table surface imply a window light source to the left, but the subject's face is lit from the right with no visible fill light or reflector.
- Generated portraits where the catchlight shape and position differ between the two eyes, suggesting each eye was rendered with a different virtual environment map.
- Temporal inconsistencies where a shadow's direction shifts between frames even though neither the subject nor the camera has moved, indicating per-frame lighting re-estimation rather than consistent scene simulation.
How Forensic Systems Analyse Lighting Consistency
The ClipForensics forensic modules include a lighting-consistency analyser that examines shadow geometry, specular highlight positions, and estimated illumination direction across regions of each frame and across time. The module produces a per-frame consistency score that highlights frames or regions where lighting appears physically inconsistent.
As with all forensic signals, lighting analysis is most useful when combined with other modules. An unusual lighting score on its own might reflect genuinely unusual but authentic lighting conditions (coloured gels, mixed indoor/outdoor sources, artistic post-processing). The multi-module pipeline fuses lighting signals with motion, compression, and frequency-domain data for a more reliable overall assessment.
Lighting-Based Detection Signals and Reliability
| Signal | What It Reveals | Reliability | Caveats |
|---|---|---|---|
| Shadow direction inconsistency | Multiple incompatible light source directions | High | Complex real lighting (e.g., stage lighting) can produce unusual shadow patterns |
| Catchlight mismatch | Different reflections in left vs right eye | High | Requires sufficiently high resolution to resolve corneal reflections |
| Specular highlight misalignment | Highlights imply different light positions on different objects | Moderate–High | Objects with different surface curvatures can naturally have non-obvious highlight positions |
| Facial illumination mismatch | Face lit from a different direction than the environment | Moderate–High | On-camera fill flash or reflectors can create legitimate mismatches |
| Missing or extra shadows | Objects without shadows or shadows without objects | High | Overexposed or very diffuse lighting can make shadows invisible in authentic footage |
| Temporal shadow drift | Shadow direction changes without corresponding scene motion | High | Slowly moving clouds or dimmer switches can cause gradual legitimate changes |
Limitations
Lighting-based detection is powerful but not infallible. Scenes with flat, diffuse lighting (overcast skies, shadowless studio setups) provide fewer lighting cues for analysis. As generative models incorporate better lighting simulation — potentially including physics-based rendering — the gap between synthetic and authentic lighting signatures may narrow over time. For a broader discussion, see our page on detection limitations.
Frequently Asked Questions
Can lighting analysis alone confirm a video is a deepfake?
No. Lighting anomalies indicate that the illumination in a scene may not be physically consistent, but they do not prove manipulation. Unusual real-world lighting setups — theatrical stages, mixed colour-temperature environments, or artistic post-processing — can produce similar anomalies. Lighting results should always be evaluated alongside other forensic signals.
What resolution is needed for catchlight analysis?
Meaningful catchlight analysis typically requires the eye region to be at least 40–60 pixels wide. Lower-resolution footage — or footage where the subject is far from the camera — may not provide enough detail for reliable corneal reflection comparison.
Do AI generators always fail at lighting?
Not always. Recent models have made significant progress in producing plausible lighting, especially in simple single-source environments. Complex multi-source scenes, dynamic lighting, and close-up facial illumination remain more challenging for generators, but improvements are ongoing. Detection tools must continually evolve in response.
How does ClipForensics handle scenes with intentionally unusual lighting?
The system flags statistically unusual lighting patterns but does not automatically classify them as manipulations. Analysts can review the flagged regions in the context of the full scan report, which includes results from all forensic modules. A scene with unusual lighting but consistent motion and compression signals is less likely to be synthetic than one where multiple modules flag anomalies.
Can I test lighting analysis on my own videos?
Yes. Upload a video through the upload tool to receive a full forensic report that includes lighting consistency scores. For additional context on interpreting results, see our articles on optical flow analysis and biomechanical motion detection.
