Why Re-Encoded Videos Are Harder to Analyse

In forensic video analysis, the single most common obstacle investigators encounter is re-encoding. A video that has been decoded and re-encoded — whether deliberately by an attacker seeking to destroy forensic evidence, or incidentally through social media platform processing, messaging app transcoding, or casual editing — loses forensic information with each encoding generation. This article examines the precise mechanisms by which video re-encoding degrades forensic signals, introduces the concept of "forensic signal half-life," and identifies which analytical techniques remain viable after multiple encoding generations. Understanding these dynamics is essential for any investigator working with video evidence in the era of synthetic media.

What Happens During H.264/H.265 Encoding

To understand why re-encoding destroys forensic signals, we must first understand what video encoding does to pixel data. The H.264 (AVC) and H.265 (HEVC) codecs — which together account for the overwhelming majority of video on the internet — use a multi-stage pipeline that fundamentally transforms the original pixel values.

Discrete Cosine Transform (DCT)

The first stage divides each frame into blocks (typically 4×4, 8×8, or 16×16 pixels for H.264; variable block sizes up to 64×64 for H.265) and applies the Discrete Cosine Transform (DCT) to convert each block from the spatial domain to the frequency domain. The DCT represents the block's pixel values as a weighted sum of cosine basis functions at different frequencies. Low-frequency components represent gradual changes (smooth gradients, average colour), while high-frequency components represent sharp edges, fine texture, and noise.

This transformation is mathematically lossless — the DCT is invertible, and the original pixel values could be perfectly reconstructed from the DCT coefficients. The information loss occurs in the next stage.

Quantization

Quantization is the stage where information is permanently discarded. Each DCT coefficient is divided by a corresponding value from a quantization table and rounded to the nearest integer. The quantization table values determine how aggressively each frequency component is compressed — higher values mean more aggressive rounding, which discards more information but produces smaller file sizes. Low-frequency coefficients typically receive smaller quantization values (preserving more information), while high-frequency coefficients receive larger values (discarding fine detail and noise).

The quantization table is the encoder's primary quality control mechanism, and it is also a critical forensic artefact. Different encoders, quality settings, and software packages produce different quantization tables. The compression archaeology module analyses these tables to determine the encoding software and quality settings used to produce a video — information that can reveal whether a video has been re-encoded, what software was used, and whether the compression profile is consistent with the video's claimed origin.

Motion Estimation and Compensation

Video codecs exploit temporal redundancy through motion estimation. Instead of encoding every frame independently, the encoder identifies blocks in the current frame that can be predicted from blocks in reference frames (previous or future frames). The motion vectors — describing the displacement between the matching blocks — and the residual (the difference between the predicted and actual blocks) are encoded instead of the full frame.

Motion vectors are a rich forensic signal. The encoder's motion estimation algorithm produces characteristic patterns that reflect the search strategy, block matching algorithm, and subpixel interpolation method used. Different encoders produce different motion vector distributions for the same content. Deepfake generators that produce their own video output use their built-in encoder, which creates a motion vector distribution characteristic of that specific generation pipeline.

In-Loop Filtering

After quantization, H.264 applies a deblocking filter to reduce visible blocking artefacts at macroblock boundaries. H.265 adds a Sample Adaptive Offset (SAO) filter that further reduces artefacts. These in-loop filters modify the reconstructed frame before it is stored as a reference for future motion compensation, meaning their effects propagate through subsequent frames. The specific filtering behaviour leaves subtle fingerprints in the output that differ between encoders and encoder implementations.

Why Re-Encoding Overwrites Quantization Tables

When a video is decoded and re-encoded, the decoder first reverses the encoding process: it dequantises the DCT coefficients (multiplying by the original quantization values), applies the inverse DCT to reconstruct pixel values, applies motion compensation to reconstruct inter-predicted frames, and applies the deblocking filter. The result is a set of pixel frames that approximate (but do not exactly match) the original uncompressed frames.

The re-encoder then takes these decoded frames and encodes them from scratch — performing block partitioning, DCT, quantization, motion estimation, and in-loop filtering with its own parameters. The quantization tables from the second encoder completely overwrite the forensic signature of the first encoder. The motion vectors are recalculated using the second encoder's motion estimation algorithm, losing the first encoder's characteristic patterns. The in-loop filtering is applied using the second encoder's parameters.

This is why compression forensics is fundamentally a first-generation technique — it can identify the most recent encoder with high confidence, but it cannot reliably identify encoders in the chain before the most recent one. Each re-encoding pass overwrites the previous pass's fingerprints, like recording over a cassette tape.

How Motion Vectors Are Recalculated

The loss of motion vector information during re-encoding deserves special attention because motion vectors are one of the most informative forensic signals in first-generation video. When a video is originally encoded, the motion estimation algorithm searches for the best-matching block in reference frames, producing motion vectors that reflect both the actual motion in the scene and the encoder's search strategy (diamond search, hexagonal search, exhaustive search, etc.).

In a deepfake, the motion vectors may exhibit anomalies: the generator's built-in encoder may use a different motion estimation strategy than the claimed recording device, or the face region may have motion vectors inconsistent with the background due to the face being generated independently. These motion vector anomalies can be powerful indicators of manipulation.

When the video is re-encoded, the new encoder performs its own motion estimation on the decoded frames. The resulting motion vectors reflect the new encoder's search strategy applied to the decoded content — they retain no memory of the original motion estimation. Any forensically significant anomalies in the original motion vectors are overwritten. The re-encoded video's motion vectors appear normal and internally consistent, because they were generated by a single encoder operating on legitimate (if previously decoded) pixel data.

Error Level Analysis Across Encoding Generations

Error Level Analysis (ELA) works by exploiting the principle that different regions of an image have different compression histories. In a first-generation JPEG or video frame, regions that were heavily compressed (high quantization) show different error levels when re-compressed at a uniform quality than regions that were lightly compressed. In a manipulated image, the pasted or generated region has a different compression history than the surrounding authentic content, creating a visible boundary in the ELA map.

Each re-encoding generation degrades ELA's effectiveness. The first re-encoding applies uniform compression to the entire frame, partially normalising the error levels across regions with different histories. The second re-encoding further normalises the distribution. After three or four encoding generations, the error level map becomes nearly uniform — the compression history of every region has been overwritten enough times that the original differences are indistinguishable from encoding noise.

For investigators, this means ELA is most reliable on first-generation video and becomes progressively less informative with each re-encoding. When social media platforms re-encode uploaded video (as most do), a single upload-and-download cycle can already significantly degrade ELA sensitivity. Video that has been shared across multiple platforms — uploaded to one, downloaded, uploaded to another — may have undergone enough re-encoding to render ELA effectively useless.

The Concept of Forensic Signal Half-Life

We introduce the concept of "forensic signal half-life" as a practical framework for reasoning about signal degradation. Just as radioactive isotopes have characteristic half-lives — the time required for half the atoms to decay — each category of forensic signal has a characteristic half-life measured in encoding generations rather than time units.

Short half-life (1–2 generations): Quantization table signatures, motion vector distributions, PRNU patterns, noise floor characteristics, and encoder-specific in-loop filter artefacts. These signals are largely destroyed by a single re-encoding pass.

Medium half-life (3–5 generations): Error Level Analysis boundaries, macroblock alignment ghosts (residual artefacts at the block boundaries of a previous encoder), frequency-domain architectural fingerprints (GAN checkerboard patterns), and compression level inconsistencies between regions. These signals degrade progressively but may remain detectable through several re-encoding generations if the original signal was strong.

Long half-life (10+ generations or indefinite): Visual-domain artefacts (facial rendering anomalies, teeth rendering errors, incorrect specular highlights), physics violations (shadow inconsistencies, impossible reflections, implausible depth relationships), temporal inconsistencies (biomechanically impossible facial movements, unrealistic motion blur), and semantic anomalies (incorrect anatomical details, physically impossible scenarios). These signals survive re-encoding because they exist in the visual content, not in the file's technical structure.

This half-life framework allows investigators to rapidly assess which analytical techniques are likely to be productive for a given piece of evidence, based on its estimated encoding history. The multi-signal analysis pipeline implements this framework by adjusting signal weights based on detected re-encoding evidence.

Signals with the Longest Survival

The forensic signals that survive the most re-encoding generations are those embedded in the visual content itself — properties of what is depicted in the frames, rather than properties of how the frames are encoded. These signals are robust because re-encoding only alters how pixel values are stored, not what they represent.

Visual Artefacts

Deepfake generators produce characteristic visual artefacts that survive any number of re-encoding generations: inconsistent skin texture between the generated face and the original head/body, boundary artefacts where the face mask blends with the background, unnaturally smooth skin in regions where real skin would show pores and fine wrinkles, and rendering errors in challenging regions like hair boundaries, ear geometry, and the inner mouth. These artefacts are pixel-level features that are preserved through re-encoding because the encoder faithfully reproduces the visual content (that is, after all, its purpose).

Physics Violations

Violations of physical laws cannot be removed by re-encoding. If a deepfake contains an impossible shadow — cast in a direction inconsistent with the scene's lighting — that shadow will be faithfully reproduced through any number of re-encoding passes. Similarly, incorrect reflections in eyeglasses or the corneal surface, physically implausible depth-of-field effects, and inconsistent perspective geometry are all signals that exist in the content domain and survive indefinitely.

These physics-based signals represent the most durable category of forensic evidence, but they are also the hardest to analyse automatically. Detecting shadow direction inconsistencies, for example, requires estimating the scene's light source positions from the available evidence — a complex inverse-rendering problem. This is an area where automated forensic analysis must be complemented by expert human review.

Temporal Inconsistencies

Temporal artefacts in deepfake video — unnatural blink patterns, lip movements that don't quite match the audio, micro-expressions that appear and disappear too quickly, and head motion that violates biomechanical constraints — are properties of the content's temporal evolution, not of the encoding. Re-encoding may slightly alter the visual fidelity of individual frames, but it preserves the temporal relationships between frames. A lip movement that is out of sync with the audio in the first-generation deepfake will remain out of sync through any number of re-encoding generations.

Detecting Re-Encoding: Forensic Archaeology

Before assessing a video's authenticity, investigators should determine its encoding history — how many times it has been re-encoded and with what parameters. Several techniques can reveal re-encoding.

Double quantization artefacts. When a video is encoded, decoded, and re-encoded, the double application of quantization can produce characteristic "ghost" artefacts in the DCT coefficient distributions. These artefacts — periodic patterns in the histogram of quantization indices — can indicate that the video has been encoded at least twice, even when the first encoder's quantization tables have been overwritten.

Macroblock alignment inconsistencies. If the first and second encoders use different macroblock grids (offset by one or more pixels), the re-encoded video may contain subtle artefacts at the boundaries of the first encoder's macroblock grid. These "grid ghosts" are difficult to produce through any process other than re-encoding and serve as strong evidence of multiple encoding generations.

Bitrate and quality inconsistencies. A video that has been re-encoded often shows a mismatch between its actual visual quality and its compression parameters. A video encoded at high bitrate but exhibiting quality artefacts consistent with prior low-bitrate encoding suggests a re-encoding chain where the first pass was more aggressive than the second.

The compression history analysis module performs these checks automatically, providing investigators with an estimated encoding generation count and the likely parameters of the most recent encoding.

Practical Implications for Investigators

Understanding the effects of re-encoding on forensic signals has several practical implications for how investigations should be conducted.

Obtain the earliest available copy. The most important procedural step is to obtain the version of the video with the fewest encoding generations. For social media content, this means requesting the original upload from the platform (which may require a legal process) rather than working with a downloaded copy that has been re-transcoded. Each generation of re-encoding degrades forensic signals, so the earliest copy provides the most analytical information.

Assess encoding history before selecting techniques. Before applying forensic techniques, estimate the video's encoding history using the methods described above. This estimate should guide the selection and weighting of analytical techniques. Applying compression forensics to a heavily re-encoded video wastes analytical resources and may produce misleading results; applying temporal and visual-domain analysis is far more productive.

Report confidence in context. Forensic reports should explicitly state the estimated encoding history and its impact on analytical confidence. A high-confidence detection result from a first-generation video carries different evidentiary weight than a moderate-confidence result from a video that has been re-encoded multiple times. The detection limitations documentation provides guidance on how to communicate these nuances to non-technical stakeholders.

Combine automated and manual analysis. For re-encoded video, the signals with the longest survival — visual artefacts, physics violations, temporal inconsistencies — are also the signals that benefit most from expert human interpretation. Automated tools can flag anomalies, but an experienced analyst can assess whether a flagged shadow inconsistency is a genuine physics violation or a compression artefact. The combination of automated screening and expert review produces the most reliable results for re-encoded content.

The Broader Challenge

Re-encoding is not just a technical challenge — it is an ecosystem challenge. The modern video distribution ecosystem — social media platforms, messaging apps, content aggregators — routinely re-encodes video as a standard part of content processing. This means that every video shared online undergoes at least one re-encoding pass, and videos that spread virally across multiple platforms may undergo several. The forensic community cannot change this ecosystem behaviour; instead, forensic techniques must evolve to remain effective within it.

Content provenance standards like C2PA offer a complementary approach by attaching cryptographic provenance metadata that survives re-encoding (because it is stored as a sidecar or in container-level metadata rather than in the pixel data). Adoption of these standards by platforms and devices would provide a provenance chain that is independent of the pixel-level forensic signals, offering an additional evidence dimension that re-encoding cannot degrade. Until such standards are universally adopted, however, pixel-level forensic analysis remains the primary tool for assessing video authenticity, and understanding the effects of re-encoding on that analysis is essential for every practitioner in the field.