Case study · AI / Explainable computer vision
Explainable Deepfake Detection
A deepfake detector that doesn't just say real or fake, it highlights the face regions behind its verdict.
Why I built it
Most deepfake detectors return a single probability and nothing else. That is hard to trust, and harder to defend when a journalist, analyst or moderator has to justify a decision.
For our final year project, my team set out to build a detector whose reasoning can be inspected: if the model calls a face fake, it should also show which parts of the face drove that call.
The challenge was keeping the accuracy of a modern CNN classifier while adding an explanation layer a human reviewer can agree or disagree with.
Goals
- Classify uploaded images and videos as real or fake
- Show a visual explanation (heatmap) alongside every verdict
- Handle video by sampling frames instead of processing every one
- Deliver it through a dashboard that feels like an investigation tool, not a demo
How it works
- 01
Face detection
MTCNN finds and crops the most prominent face from an image, or from up to 15 evenly spaced frames of a video, resized to 224x224 for the classifier.
- 02
Classification
A fine-tuned EfficientNet with a custom classification head scores each face crop. Scores are averaged across frames to give one video-level verdict.
- 03
Explainability
Grad-CAM hooks into the last convolutional block and produces a heatmap over the highest-confidence face, showing which regions influenced the prediction.
- 04
API and dashboard
A Flask endpoint runs the pipeline and returns the verdict and heatmap to a React dashboard with drag-and-drop upload and live pipeline-stage feedback.
What it does
Image and video input
Upload a photo or a short clip and get a single verdict.
Grad-CAM heatmaps
See the facial regions that pushed the model toward real or fake.
Face-focused pipeline
MTCNN crops keep the classifier looking at the face, not the background.
Frame-level aggregation
Averaging over sampled frames makes video verdicts less fragile than a single frame.
CPU / GPU fallback
Runs on CUDA when available and falls back to CPU automatically.
Forensic dashboard
A React interface that walks the reviewer through each stage before showing the result.
What went wrong, and the fix
A bare confidence score is not evidence a reviewer can question.
Added Grad-CAM so every verdict ships with a heatmap that can be checked visually.
Videos are too long to analyse frame by frame.
Sampled up to 15 evenly spaced frames, which keeps coverage proportional while keeping inference fast.
Inconsistent face crops hurt classifier input quality.
Used MTCNN with a margin around the face so every crop is aligned and sized the same way.
Temporal analysis is more than a heuristic.
The dashboard's temporal score is currently derived from the variance of per-frame predictions, and I label it as a proxy rather than a dedicated sequence model.
Gallery
What I learned
- Explainability changes how people use a model: a heatmap turns a number into something you can discuss.
- Splitting the system into detection, classification and explanation made each stage easier to test and debug.
- Being upfront about what a heuristic score can and cannot show matters as much as the model itself.
What's next
- Replace the variance-based temporal score with a real sequence model (LSTM or 3D-CNN)
- Benchmark accuracy on public datasets such as FaceForensics++ or Celeb-DF
- Analyse every face in multi-person videos, not just the most prominent one
Have something in mind?
Tell me what you're building. I'll reply with questions, a rough plan and a realistic timeline.
Start a conversation