Case study · AI / Computer vision
Tom & Jerry Emotion Detection
A PyTorch neural network that recognises emotions on cartoon faces using transfer learning.
Why I built it
Emotion recognition models are usually trained on real human faces. Cartoon faces have exaggerated features, simplified eyes and mouths, and flat shading, so models built for photographs often misread them.
Collecting a huge cartoon dataset is not realistic for a solo project, so the real question was how well a compact pretrained network can adapt to stylised faces with limited data.
I chose Tom and Jerry because the characters show strong, easy-to-label expressions, which makes them a good test case for cartoon emotion classification.
Goals
- Classify the emotion shown on a cartoon face image
- Reuse a pretrained backbone instead of training from scratch
- Keep the model small enough to run on modest hardware
- Wrap the model in a simple web interface so anyone can test it
How it works
- 01
Data collection and labelling
Collected cartoon face images and sorted them into emotion classes. I cleaned out unclear frames and checked the class counts so one emotion did not dominate the training set.
- 02
Transfer learning model
Used MobileNetV2, pretrained on ImageNet, as a feature extractor. Its general visual features (edges, shapes, textures) carry over to cartoons, so only a new classification head needed to be trained in PyTorch.
- 03
Training and evaluation
Fine-tuned the model on the labelled set and evaluated it on held-out images it had never seen, to check it generalised instead of memorising the training data.
- 04
Serving through Flask
Exposed the trained model through a Flask app. A user uploads an image, the backend preprocesses it, runs inference, and returns the predicted emotion to a lightweight HTML/CSS/JS front end.
What it does
Image upload
Upload a cartoon face and get an emotion prediction in the browser.
Transfer learning
MobileNetV2 backbone with a custom classifier head built in PyTorch.
Lightweight
A small model that runs comfortably on modest hardware, no GPU needed for inference.
Held-out evaluation
Tested on unseen images to check the model generalises beyond its training data.
Web interface
Flask backend with a simple front end, so the model is usable without any code.
Open source
Full code is public on GitHub for anyone to read, run or extend.
What went wrong, and the fix
A small dataset makes overfitting likely.
Kept the pretrained backbone frozen, trained only the new head, and used augmentation so the model saw more variation.
Exaggerated cartoon expressions can look similar across classes.
Cleaned ambiguous frames from the dataset and checked results per emotion rather than trusting one overall score.
Real-face emotion models do not transfer to cartoons.
Trained on cartoon-specific images instead of reusing a model built for human faces.
Gallery
What I learned
- Transfer learning lets a small dataset go a long way when the pretrained features are reusable.
- Data quality and class balance mattered more than model tweaks.
- Serving a model behind a simple Flask route is what turns an experiment into something people can try.
What's next
- Add more characters and emotion classes
- Add face detection so full scenes can be analysed, not just cropped faces
- Show a confidence score with each prediction
Have something in mind?
Tell me what you're building. I'll reply with questions, a rough plan and a realistic timeline.
Start a conversation