Multi-Object Tracking
Real-time tracking of people and vehicles in video with YOLOv5, Deep SORT and OC-SORT
Supervisor: Prof. Muhammad Usman Akram
Computer Vision semester project, National University of Sciences and Technology (NUST), Islamabad, 2025
Background and Goals
Object detection finds objects in a single image. Many real-world systems also need to know which object is which over time: the same pedestrian walking across a street, or the same car moving through traffic. This is object tracking. It keeps a consistent identity for each object across video frames, and it is the basis of autonomous driving, surveillance, sports analytics and gesture-based interfaces.
The goal of this project was to build a multiple object tracking (MOT) system that detects people and vehicles in a busy street video, gives each one a unique ID, and keeps that ID as they move through the scene.
Two central questions:
(i) Can a modern object detector be combined with a tracking algorithm to follow many objects at once in a crowded scene?
(ii) Can each object keep the same identity across frames, even as people pass close to one another?
Methodology
The system follows the tracking-by-detection approach: detect objects in every frame, then link those detections across frames.
- Object detection: YOLOv5 (PyTorch) finds objects in each frame. Detections are filtered by class (person, car, truck, bus and motorcycle) and by confidence.
- Motion prediction: a Kalman filter predicts where each tracked object will be in the next frame.
- Tracking algorithms: two trackers were used.
- Deep SORT adds an appearance model, a learned description of how each object looks, which helps tell similar objects apart and recover them after short occlusions.
- OC-SORT (Observation-Centric SORT) relies on the detections themselves to correct the motion model. This makes it more robust when objects are hidden for a while or move in non-linear ways.
- Data association: the Hungarian algorithm matches new detections to existing tracks, so each object keeps its ID.
- Visualisation: each frame is drawn with bounding boxes, class labels, track IDs and a live FPS counter, and the annotated video can be saved.
The system was tested on MOT16-03, a crowded street scene from the MOT16 benchmark, using Python, PyTorch, OpenCV, YOLOv5, Deep SORT and OC-SORT.
Key findings
- Many objects tracked at once. People, cars and motorcycles in a dense street scene are detected and followed together.
- Stable identities. Each object keeps a unique ID across frames, which is the core requirement of multiple object tracking.
- Correct classes. Object classes stay consistent as they are tracked.
The same approach can be applied to traffic monitoring, crowd analysis, sports analytics and surveillance.