Inside a Deepfake Detection Investigation

Forensic analysis of suspected deepfake video is not a single test — it is a structured investigative workflow comprising twelve distinct phases, each building on the findings of the previous phases and each producing independently documented results. This article walks through the complete operational workflow used in professional deepfake detection investigations, from initial evidence intake through final report generation. The methodology described here reflects current best practice in digital forensic analysis and is designed to produce results that are defensible in legal, journalistic, and intelligence contexts.

Phase 1: Evidence Intake and Chain of Custody

Every investigation begins with evidence intake — the formal process of receiving the video under analysis and establishing a documented chain of custody. This phase is procedural rather than analytical, but it is essential for the evidentiary value of the results. Without a documented chain of custody, even the most rigorous forensic analysis may be inadmissible in legal proceedings or dismissed by stakeholders.

During intake, the investigator records the source of the video (who provided it, when, through what channel), the claimed provenance (where and when the video was allegedly recorded, by whom, on what device), and the analytical question being asked (is this video authentic, has it been manipulated, is the depicted person actually the claimed individual). The original file is cryptographically hashed (SHA-256) and the hash is recorded as the evidence identifier. All subsequent analysis is performed on a working copy; the original file is preserved unmodified.

The intake phase also captures the file's container-level properties: file format, file size, duration, resolution, frame rate, codec information, and any immediately visible metadata. These properties provide the initial context for the investigation and may reveal obvious anomalies — for example, a video claimed to be raw smartphone footage but encoded in a professional broadcast codec.

Phase 2: Metadata Extraction and Analysis

The second phase performs comprehensive metadata extraction using tools that parse all available metadata fields: EXIF, XMP, ICC profiles, encoder version strings, creation and modification timestamps, GPS coordinates (if present), device model identifiers, and any embedded content credentials (C2PA manifests, IPTC headers).

Metadata analysis focuses on consistency and plausibility. Does the creation timestamp align with the claimed recording time? Is the device model identifier consistent with the video's technical characteristics (resolution, codec, frame rate)? Does the encoder version string match a known encoder, and is it the encoder expected for the claimed recording device? Are there inconsistencies between different metadata fields — for example, a creation date in the future, or a GPS location inconsistent with the claimed recording location?

Metadata absence is itself informative. A video that claims to be raw smartphone footage but contains no device model identifier, no GPS data, and no embedded thumbnail may have been stripped of metadata — a processing step that can indicate either innocent post-processing or deliberate evidence removal. The metadata analysis module documents both the presence and absence of expected metadata fields.

Phase 3: Compression Archaeology

Compression archaeology examines the video's encoding at the technical level to determine its processing history. This phase analyses quantization tables, macroblock structures, motion vector distributions, keyframe intervals, and encoding profile parameters to answer several questions: what encoder was used, at what quality settings, how many times has the video been encoded, and are there inconsistencies in the compression that suggest manipulation?

A key technique in this phase is double compression detection — identifying whether the video has been encoded, decoded, and re-encoded. Double compression leaves characteristic artefacts in the DCT coefficient distributions that can reveal the quantization parameters of the first encoding, even after the second encoding has overwritten the quantization tables.

For deepfake detection specifically, compression archaeology can reveal whether the video's encoding profile is consistent with its claimed origin. A deepfake produced by a generator with a built-in encoder will have a compression profile that differs from the profile of the claimed recording device. If the video has been re-encoded to disguise this, double compression artefacts may still reveal the original encoding.

Phase 4: Visual Artefact Scanning

This phase examines individual frames for visual artefacts associated with synthetic generation. The analysis is both automated (using trained classifiers that scan for known artefact patterns) and manual (expert visual inspection of regions flagged by the automated scan).

Key artefacts targeted include: boundary discontinuities at the edge of generated regions (face mask boundaries), inconsistent texture resolution between the face and surrounding areas, rendering errors in geometrically complex regions (teeth, ear canals, hair strands, jewellery), unnatural skin smoothness in regions where real skin would show fine detail, asymmetric rendering of naturally symmetric features, and incorrect specular reflection patterns on the skin surface.

The automated scan produces a spatial map of anomaly likelihood across each frame, highlighting regions that warrant closer expert inspection. This targeted approach is more efficient than exhaustive manual frame-by-frame review and ensures that subtle artefacts in non-obvious locations (ear geometry, neck-chin boundary, hairline) are not overlooked.

Phase 5: Temporal Consistency Analysis

Temporal analysis examines the consistency of the video's content across frames. Real video exhibits smooth, physically consistent evolution of all visual elements; deepfakes frequently introduce temporal inconsistencies that are invisible in individual frames but detectable in frame sequences.

The analysis tracks hundreds of facial landmarks across the video's duration, comparing their trajectories to biomechanical models of natural facial movement. Anomalies include: sudden position shifts in landmarks that should move smoothly, periodic oscillations in facial geometry that correspond to the generator's frame processing cycle, inconsistent motion blur (frames where fast head movement produces sharp features, or slow movement produces blurred features), and temporal flickering in skin texture or specular highlights.

The temporal consistency engine also analyses the relationship between facial motion and background motion. In real video, camera movement affects both the face and the background consistently; in a face-swap deepfake, the generated face may exhibit slightly different motion characteristics than the background, revealing the composition boundary.

Phase 6: Face Manipulation Detection

This phase applies specialised deep-learning classifiers trained specifically to detect face manipulation — face swaps, face reenactment, expression transfer, and fully synthetic face generation. These classifiers operate on multiple scales: pixel-level analysis of facial regions, patch-level analysis of texture consistency, and holistic analysis of facial geometry.

Modern face manipulation detectors use attention mechanisms to identify the most discriminative regions in the face — areas where the difference between real and generated content is most pronounced. These regions often include the boundaries of the generated mask, the inner corners of the eyes, the nostrils, and the boundary between the teeth and the inner mouth. By focusing analytical attention on these discriminative regions, the classifiers achieve higher sensitivity than whole-face approaches.

The phase also includes identity verification — comparing the depicted face against known reference images of the claimed individual to detect face-swap manipulations. This comparison goes beyond simple face recognition to examine whether fine facial details (specific moles, freckles, scar patterns, wrinkle configurations) are consistent between the video and the reference material.

Phase 7: Audio-Visual Synchronisation

For video containing speech, audio-visual synchronisation analysis examines the relationship between the visible lip movements and the audio waveform. In authentic video, lip movements and speech sounds are produced by the same physical process (articulatory phonetics) and are therefore tightly coupled. In deepfake video, the face may be generated independently of the audio, producing subtle synchronisation errors.

The analysis measures the temporal alignment between phoneme boundaries in the audio stream and the corresponding viseme (visual speech unit) poses in the video. Real speech exhibits characteristic timing relationships between auditory and visual components — for example, the lip closure for bilabial consonants (/p/, /b/, /m/) should precede the acoustic burst by a consistent interval. Deepfake generators may fail to reproduce these precise timing relationships, producing lip movements that are approximately but not exactly synchronised with the audio.

This phase also examines the physical consistency of the audio environment. The room acoustics (reverberation, echo patterns) audible in the audio track should be consistent with the visual environment depicted in the video. An intimate close-up shot in what appears to be a small room, but with audio reverb characteristics of a large open space, is an inconsistency that suggests the audio and video may have been combined from different sources.

Phase 8: Voice Synthesis Detection

Modern deepfakes increasingly use synthesised voice to match the face manipulation, creating a complete audiovisual fabrication. Voice synthesis detection analyses the audio track independently to determine whether the voice is human-produced or machine-generated.

Key analytical signals include: spectral characteristics (synthetic voices often exhibit smoother spectral envelopes than natural speech, missing the micro-variations caused by the physical vocal tract), prosodic patterns (pitch contour, speaking rate variation, and emphasis patterns that may be statistically regular in ways that natural speech is not), breathing patterns (natural speech includes breath intake at phrase boundaries; synthetic speech may omit or incorrectly place breath sounds), and noise floor characteristics (the background noise in synthetic speech is often too clean or too uniform compared to a real recording environment).

The voice analysis module also performs speaker verification — comparing the voice in the video against known voice samples of the claimed speaker to detect voice cloning. This comparison examines both acoustic features (formant frequencies, spectral tilt, jitter, shimmer) and speaking style features (habitual phrases, speech rate patterns, characteristic prosody).

Phase 9: Provenance Verification

Provenance verification attempts to trace the video's origin and distribution history through non-forensic means. This phase examines the platforms where the video appeared, the accounts that posted it, the timing and pattern of its distribution, and any available platform-specific metadata (upload timestamps, original filenames, account creation dates).

Key provenance signals include: whether the video appeared first on a single platform or was simultaneously posted across multiple platforms (simultaneous multi-platform seeding is characteristic of coordinated disinformation operations), whether the posting account has a credible history or was recently created, whether the video's distribution pattern shows organic viral spread or artificial amplification, and whether any content credentials (C2PA manifests, blockchain attestations) can be verified.

Reverse image and video search is used to identify earlier instances of the video or its component elements. Finding the original background video (before the face was swapped) or the source material used to train the face generator provides strong evidence of manipulation that complements the technical forensic analysis.

Phase 10: Multi-Signal Correlation

The tenth phase integrates the findings from all previous phases into a coherent analytical picture. Each phase produces independent signals — metadata anomalies, compression inconsistencies, visual artefacts, temporal anomalies, synchronisation errors, voice synthesis indicators, provenance flags — and multi-signal correlation examines whether these signals form a consistent pattern.

The power of multi-signal correlation lies in its resistance to targeted anti-forensic techniques. An attacker who focuses on eliminating visual artefacts may neglect temporal consistency. An attacker who addresses audio-visual synchronisation may introduce compression anomalies. Fooling all analytical dimensions simultaneously is extremely difficult, and the correlation analysis is designed to detect even partial signal convergence.

The signal fusion engine applies weighted combination of signals, with weights determined by the estimated reliability of each signal category for the specific video under analysis. For example, if the video is determined to have been re-encoded multiple times, compression-based signals receive lower weight, while visual-domain and temporal signals receive higher weight.

Phase 11: Confidence Assessment

Rather than producing a binary "real" or "fake" verdict, the investigation produces a confidence assessment that quantifies the strength of evidence for and against manipulation. This assessment takes the form of a probability distribution over possible explanations for the observed signals: authentic video with expected characteristics, authentic video with unusual but explicable characteristics (unusual lighting, non-standard equipment), or manipulated video with varying degrees of manipulation complexity.

The confidence assessment explicitly accounts for the limitations of each analytical technique applied. If a technique was unable to produce a meaningful result (for example, compression analysis on a heavily re-encoded video), this is documented as a limitation rather than as evidence of authenticity. The absence of evidence is not evidence of absence — a principle that is critical for maintaining analytical rigour.

The assessment also documents the base rate: how common are deepfakes with the observed characteristics in the current threat landscape? A video that matches the profile of a known deepfake generation tool (specific artefact patterns, characteristic encoding profile) carries different evidentiary weight than a video with anomalies that could be explained by either manipulation or unusual legitimate processing.

Phase 12: Report Generation

The final phase produces a structured forensic report that documents every phase of the investigation, including: the evidence identification and chain of custody, the analytical methods applied and their results, the signals detected and their assessed reliability, the multi-signal correlation analysis, the confidence assessment with explicit uncertainty quantification, and the limitations of the analysis.

The report is structured for multiple audiences. The executive summary provides a non-technical overview of the findings for decision-makers. The technical body provides detailed results for peer review by other forensic analysts. The appendices include raw data, configuration parameters, and tool versions for reproducibility.

A critical element of the report is the limitations section. This section explicitly documents what the analysis can and cannot conclude, including the impact of re-encoding on signal availability, the known false-positive and false-negative rates of the classifiers used, and any analytical techniques that were not applied (and why). This transparency is essential for the credibility of forensic analysis — particularly in adversarial contexts where the report's conclusions will be challenged.

The Workflow in Practice

In operational practice, the twelve phases are not always executed strictly sequentially. Preliminary results from early phases may redirect the focus of later phases — for example, the discovery of screen-recording artefacts in Phase 3 may cause Phase 4 and 5 to receive elevated priority, while Phase 3's own results are discounted in the final assessment. Similarly, a strong voice synthesis detection signal in Phase 8 may prompt additional scrutiny of audio-visual synchronisation in Phase 7.

The automated analysis pipeline executes the automated components of all phases in parallel, producing a preliminary assessment within minutes. Expert analysts then review the automated results, conduct manual inspection of flagged regions and timeframes, and perform the integration and confidence assessment phases. This hybrid approach — automated screening followed by expert analysis — provides both speed and depth, enabling rapid triage of large volumes of content while maintaining the analytical rigour required for high-stakes investigations.

Why a Structured Workflow Matters

The structured twelve-phase workflow serves several critical functions beyond analytical thoroughness. First, it provides defensibility — when the investigation's conclusions are challenged, the structured documentation of every analytical step demonstrates methodological rigour. Second, it enables reproducibility — another analyst can follow the same workflow on the same evidence and verify the results. Third, it prevents confirmation bias — by requiring systematic analysis across all signal categories, the workflow prevents investigators from reaching conclusions based on a single striking artefact while ignoring contradictory signals.

For organisations building deepfake detection capabilities, adopting a structured workflow based on these twelve phases — adapted to their specific operational context, resources, and analytical tools — is the most effective path to reliable, defensible results. The methodology is tool-agnostic: it defines what must be analysed, not which specific tool must be used at each step, allowing organisations to leverage their chosen forensic platform within a rigorous analytical framework.