In this machine-vision course project, I estimated where people and vehicles are in 3D, and how fast they’re moving, using only a phone camera without LiDAR or a stereo rig.
What I built
YOLO11n detects objects and ByteTrack keeps their identities across frames. The metric outdoor variant of Depth Anything V2 then turns each detection into a 3D position.
A recursive Kalman filter tracks motion with a six-value state (position and velocity in x, y, z). It uses wall-clock time between frames, so velocity stays consistent when the frame rate varies, and it keeps predicting through missed detections.
I calibrated the camera (a Samsung A34) with a checkerboard, and used a second phone’s IMU (a Samsung A54, via Phyphox) to sanity-check the estimated velocities.
How I chose the components
I benchmarked the detectors myself on the target laptop (RTX 3060, 6 GB):
| Detector | Measured speed |
|---|---|
| YOLOv8n | ≈ 16 FPS |
| YOLO11n | ≈ 20 FPS |
| YOLO26n | ≈ 23 FPS |
I picked YOLO11n. It wasn’t the fastest, but it gave the best balance of detection quality, installation reliability and speed. The depth model was the real bottleneck at about 0.22 s per frame (≈ 4.5 FPS).
Results and limits
The Kalman filter typically converged within three to five frames of a new detection. I tested qualitatively on crowded pavements, street traffic and mixed traffic.
The report states the main limitation. Without LiDAR or stereo ground truth, I could not measure end-to-end depth error, and metric scale needed manual calibration. The depth model’s published benchmark (0.454 m MAE on its own validation sets) does not measure this system.