Methodology
How we produce verdicts — module weights, evidence fusion, confidence calibration, and verdict thresholds.
Overview
ClipForensics produces video authenticity assessments through a multi-stage process: independent module analysis, weighted evidence fusion, and threshold-based verdict classification. This page documents each stage in detail.
We publish this methodology because we believe transparent methods produce more trustworthy results. Users should understand how their verdicts are generated.
Stage 1
Independent module analysis
Each of 15 forensic modules runs independently on the input video. Modules do not share intermediate results — they each work from the raw extracted frames, audio, and metadata.
Each module produces:
- Score: A value between 0 (high manipulation risk) and 1 (high authenticity confidence).
- Confidence: How certain the module is about its score (0 to 1).
- Evidence list: Specific findings that support the score.
- Timeline events: Findings mapped to specific timestamps.
Stage 2
Weighted evidence fusion
Module scores are combined using calibrated weights. The weight of each module reflects its reliability and discriminative power based on validation testing.
The fusion process applies two key adjustments:
Agreement bonus
When multiple independent modules reach similar conclusions (e.g., face manipulation, lip-sync, and temporal consistency all flag anomalies), the overall confidence increases beyond what any single module would produce. Independent agreement is stronger evidence than a single strong signal.
Contradiction penalty
When modules produce conflicting results (e.g., metadata suggests authenticity but face analysis suggests manipulation), the overall confidence decreases. The system is pushed toward "Inconclusive" rather than arbitrarily choosing one signal over another.
Stage 3
Verdict classification
The fused trust score maps to one of six verdict categories based on threshold ranges:
| Verdict | Trust Score Range | Meaning |
|---|---|---|
| Verified Origin | N/A (provenance-based) | Valid C2PA credentials verify content origin. |
| Likely Authentic | 0.75 – 1.00 | No significant manipulation signals detected. |
| Likely AI-Edited | 0.50 – 0.74 | Some indicators suggest AI modifications. |
| Likely AI-Generated | 0.25 – 0.49 | Multiple signals suggest synthetic origin. |
| High Manipulation Risk | 0.00 – 0.24 | Strong evidence of manipulation. |
| Inconclusive | Any (low confidence) | Insufficient evidence for determination. |
Calibration
Ongoing calibration
Module weights and verdict thresholds are calibrated against our validation dataset and updated as new generation methods emerge. See our validation study for current performance metrics.
We recognize that calibration is an ongoing process. As AI generation technology evolves, our detection methods must evolve with it.