Harris–LK Vehicle Tracker

Classical single-object vehicle tracking using Harris Corner Detection and pyramidal Lucas–Kanade Optical Flow — no deep learning, no Kalman filter.

Oblique 4K drone footage · 529 frames · 100% stability · zero manual interventions

• Quick Start
• Why Classical CV?
• Overview
• System Architecture
• Algorithm Deep-Dive
  • Harris Corner Detection
  • Pyramidal LK Optical Flow
  • Forward-Backward Verification
  • Velocity-Guided Background Filter
  • Survival-Based Redetection
  • Velocity Prediction Fallback
  • CLAHE Preprocessing
• Performance Benchmarks
  • vs OpenCV Built-in Trackers
• Results & Demo
• Project Structure
• Installation
• Usage
• Configuration Reference
• Roadmap
• Developer Guide
• Tested Environment
• Troubleshooting
• Known Limitations
• Academic Context
• Citation
• License

# 1. Clone and install
git clone https://github.com/whozahm3d/harris-lk-vehicle-tracker.git
cd harris-lk-vehicle-tracker
python -m venv venv && venv\Scripts\activate
pip install -r requirements.txt

# 2. Place your video
#    Copy your .mp4 into the video/ folder

# 3. Set your video path and bounding box in config.yaml
#    Run phase2_algorithms.ipynb to get the exact bbox coordinates

# 4. Track
python main.py --config config.yaml
#    Output saved to results/<run_timestamp>/

Full setup details — including the OpenH264 DLL requirement on Windows — are in the Installation section. Bounding box selection is covered step-by-step in Usage.

Modern trackers (SORT, DeepSORT, ByteTrack) rely on deep-learning detectors as the front-end and often treat optical flow as a secondary cue. This project takes the opposite stance: classical feature-based tracking is a design goal, not a constraint.

The motivation is threefold:

1. Interpretability. Every decision in the pipeline — which points are tracked, why a bbox shifts, when redetection fires — can be explained in closed form. There are no learned weights to inspect.
2. Zero inference cost. No GPU is required. The entire system runs on a standard laptop CPU and is deployable in environments where neural-network runtimes are unavailable.
3. Demonstrating sufficiency. Well-engineered classical methods can match deep-learning trackers in narrow, well-defined scenarios. This project validates that claim on real 4K drone footage: 100% stability, 0 manual interventions, across 809 total frames.

This project implements a complete, interpretable single-object tracker built entirely on classical computer vision primitives. Given a user-defined bounding box on a start frame, the system detects Harris corners inside the box and tracks them across subsequent frames using pyramidal LK optical flow — updating the bounding box each frame via the median displacement of surviving feature points.

Validated on two 4K test sequences — 100% tracker stability on both, across 280 and 529 frames respectively. Zero manual interventions. Zero restarts.

This loop repeats for every frame until the sequence ends, the user presses Q, or the tracker deactivates after 45 consecutive zero-point frames (~1.5 s at 30 fps).

The tracker is structured as a five-layer pipeline. Each layer has a single responsibility; no layer reaches across two layers to read or write state.

┌─────────────────────────────────────────────────────────────────────────┐
│                          harris-lk-vehicle-tracker                      │
│                                                                         │
│  ┌────────────┐    ┌──────────────┐    ┌───────────────────────────┐   │
│  │  video_io  │───▶│     core     │───▶│       visualization       │   │
│  │            │    │              │    │                           │   │
│  │ video_     │    │ harris_      │    │ display.py  (bbox/trail)  │   │
│  │ source.py  │    │ detector.py  │    │ plotter.py  (diagnostics) │   │
│  │ output_    │    │ lucas_       │    └───────────────────────────┘   │
│  │ writer.py  │    │ kanade.py    │                                     │
│  └────────────┘    │ object_      │    ┌───────────────────────────┐   │
│                    │ tracker.py   │───▶│         metrics           │   │
│  ┌────────────┐    └──────────────┘    │                           │   │
│  │   utils    │          │             │ logger.py                 │   │
│  │            │          ▼             │ performance_tracker.py    │   │
│  │ config_    │    ┌──────────────┐    └───────────────────────────┘   │
│  │ loader.py  │    │  config.yaml │                                     │
│  │ folder_    │    │  (all params)│                                     │
│  │ manager.py │    └──────────────┘                                     │
│  │ health_    │                                                         │
│  │ check.py   │                                                         │
│  └────────────┘                                                         │
└─────────────────────────────────────────────────────────────────────────┘

flowchart TD
    A([Video Source]) --> B[Decode Frame]
    B --> C[CLAHE Preprocessing]
    C --> D{First Frame?}

    D -->|Yes| E[cv2.selectROI\nManual BBox Init]
    E --> F[Harris Detection\ninside bbox]
    F --> G

    D -->|No| G[Forward LK Pass\nprev → curr]
    G --> H[Backward LK Pass\ncurr → prev]
    H --> I[F-B Error Filter\nreject > 3.5 px]
    I --> J{Speed > 3 px/f?}

    J -->|Yes| K[Background Filter\ndiscard disp < 30% velocity]
    J -->|No| L

    K --> L[Median Shift\nUpdate BBox]
    L --> M[Update Velocity EMA\nα = 0.6]
    M --> N{Survival < 25%?}

    N -->|Yes| O[Survival-Triggered\nRedetection]
    N -->|No| P{Active Points ≥ 4?}
    O --> P

    P -->|No| Q[Velocity Prediction\nFallback]
    P -->|Yes| R

    Q --> R[Update Centroid + Trail]
    R --> S{Inactive > 45 frames?}
    S -->|Yes| T([Tracker Deactivated])
    S -->|No| U([Next Frame])
    U --> B

The Harris detector identifies points where image intensity changes sharply in all directions — a necessary property for reliable optical flow estimation.

For a local window $W$ centred at pixel $(x, y)$, the auto-correlation matrix $M$ is:

$$M = \sum_{(u,v) \in W} w(u,v) \begin{bmatrix} I_x^2 & I_x I_y \ I_x I_y & I_y^2 \end{bmatrix}$$

where $I_x$ and $I_y$ are image gradients and $w(u,v)$ is a Gaussian weighting kernel. The corner response score $R$ is then:

$$R = \det(M) - k \cdot \text{tr}(M)^2 = \lambda_1 \lambda_2 - k(\lambda_1 + \lambda_2)^2$$

High $R$ → corner. Near-zero $R$ → flat region. Large negative $R$ → edge. The sensitivity constant $k = 0.04$ is the standard Shi-Tomasi value and is rarely tuned.

In this implementation:

• CLAHE-normalised grayscale is used as input so gradient magnitudes are consistent across lighting conditions
• Detection is masked to the user-defined bounding box — corners on background texture are never included in the initial pool
• quality_level = 0.005 is intentionally low to maximise the initial feature count on vehicles with limited texture (e.g. uniform car rooftops)

Vanilla Lucas–Kanade solves for the displacement $(u, v)$ that minimises the SSD of image patches between frames:

$$E(u, v) = \sum_{(x,y) \in W} \bigl[I(x, y, t) - I(x+u, y+v, t+\Delta t)\bigr]^2$$

This is valid only for small displacements. 4K drone footage contains large inter-frame vehicle motion — a 70 km/h vehicle at 30 fps moves ~30 pixels per frame at typical drone altitudes. Pyramidal LK solves this by building an image pyramid and solving from coarse to fine:

• Level 5 (coarsest): Image downsampled by $2^5 = 32\times$. A 30 px motion appears as ~1 px — well within the linear approximation range.
• Levels 4→0 (refinement): Each level refines the estimate using the previous level's solution as initialisation.

Window size [41, 41] is large by default. This captures more gradient information per point and is critical on the uniform surfaces common in drone footage (tarmac, car rooftops).

After tracking points forward from frame $t$ to $t+1$, the same points are tracked backward from $t+1$ to $t$. The round-trip error for point $i$ is:

$$\varepsilon_i = |p_i^{(t)} - \hat{p}_i^{(t)}|_2$$

where $p_i^{(t)}$ is the original position and $\hat{p}_i^{(t)}$ is the back-projected position. Points with $\varepsilon_i > 3.5$ px are discarded. This is a self-supervised consistency check — no ground truth required. In practice, nearly all valid tracking points cluster below 1 px round-trip error; degenerate matches (caused by occlusion, specular reflection, or motion blur) produce errors of 5–20 px and are cleanly separated.

Problem solved: A fast-moving vehicle shares the frame with stationary Harris corners on road markings, kerb edges, and pavement texture. These produce near-zero optical flow and contaminate the median displacement, causing bbox drift toward the background.

Mechanism: When the current velocity estimate exceeds 3 px/frame, any tracked point whose displacement magnitude falls below 30% of the current velocity is classified as a stationary background point and discarded before the median is computed:

speed = sqrt(vx² + vy²)
if speed > 3.0:
    motion_mask = displacement_magnitude > speed * 0.3
    good_new = good_new[motion_mask]

Why this threshold works: The 30% threshold creates a soft band around stationary. A point moving at 10 px/frame is only kept if it displaces more than 3 px — road markings and static texture reliably fall below this, while vehicle surface features track within a tight range of the object velocity.

Outcome: No per-video threshold tuning. The filter is off when the object is slow (preventing false rejection of valid points) and activates automatically when it's needed.

Problem solved: Scheduled redetection (every $N$ frames) wastes computation when the point cloud is healthy and fires too late when points degrade suddenly due to texture change or partial occlusion.

Mechanism: Redetection triggers only when the surviving point count falls below 25% of initial_pts — the count detected at initialisation or the most recent redetection. Harris corners are re-detected inside the current bbox estimate, fully replenishing the feature pool.

survival_ratio = len(good_new) / initial_pts
if survival_ratio < 0.25:
    redetect(curr_gray, current_bbox)

Outcome in Test 2: No redetection triggered across all 529 frames — the point cloud remained fully healthy throughout. The survival check ran every frame at negligible cost and correctly determined each time that redetection was unnecessary.

Problem solved: When a vehicle exits the frame or passes through a brief full occlusion, surviving point count can drop to zero. Without a fallback, the bbox freezes at its last known position.

Mechanism: An exponentially weighted moving average velocity estimate is maintained every frame:

$$v_t = \alpha \cdot \Delta x_t + (1 - \alpha) \cdot v_{t-1}, \quad \alpha = 0.6$$

When active point count drops critically (below min_points_per_object = 4), bbox position is updated by blending the measured displacement with the velocity estimate, weighted by how many points remain:

weight = clamp((num_points / min_points) - 1, 0.0, 1.0)
dx = weight * measured_dx + (1 - weight) * velocity_x

When zero points survive, pure velocity extrapolation advances the bbox for up to 45 frames.

Outcome: The tracker follows the vehicle to the frame boundary in Test 2 rather than stalling — critical for exit-phase sequences where point count collapses rapidly as the vehicle leaves the field of view.

Contrast Limited Adaptive Histogram Equalisation (CLAHE) partitions the image into small tiles and equalises each tile's histogram independently, with a clip limit to prevent noise amplification:

• Clip limit: 3.5 — aggressive enough to recover texture in shadows without creating haloing artifacts
• Tile grid: 4×4 — fine enough to adapt to local illumination variation at 4K resolution

CLAHE is applied to the grayscale frame before both Harris detection and LK flow computation. This ensures that gradient magnitudes are consistent across sun glare, shadow boundaries, and exposure variations that are common in outdoor drone footage.

The table below shows which engineering mechanisms were active and decisive in each test:

All profiling on: Intel Core i7 (8-core), no GPU, 4K input at native resolution.

Throughput note: The bottleneck is dual LK passes at 4K resolution with a 5-level pyramid and 41×41 window. Downscaling input to 1920×1080 cuts LK cost by ~4× and brings throughput to ~5–6 fps with minimal accuracy loss on typical drone footage.

OpenCV ships five classical single-object trackers out of the box. The question any CV-literate reviewer will ask is: why not just call cv2.TrackerCSRT_create()? The table below answers it directly.

Key architectural differences:

• Redetection. CSRT, KCF, MIL, MOSSE, and MedianFlow have no redetection mechanism — once the target texture degrades or point count collapses, they fail silently or freeze. This project triggers redetection automatically when the surviving feature count drops below 25% of the initial pool.

• Background clutter. OpenCV's built-in trackers treat the entire tracked region as signal. On top-down drone footage with road markings inside the bbox, this causes immediate drift. The velocity-guided background filter in this project discards near-stationary points before the displacement estimate is computed — a problem none of the built-ins address.

• Velocity fallback. No OpenCV built-in tracker maintains a velocity model. On exit-phase sequences (Test 2), where point count collapses as the vehicle leaves frame, every built-in tracker would freeze or report an error. This project's EMA velocity extrapolation keeps the bbox on-target for up to 45 frames without any tracked points.

• MedianFlow is the closest architectural relative — it also uses optical flow with forward-backward verification. The primary difference is that MedianFlow has no background filter, no survival-triggered redetection, and no velocity fallback, making it fragile on cluttered or high-motion drone footage.

All plots, metrics, logs, and parameter snapshots inside results/ are committed to this repository and render correctly on GitHub. Only the annotated output videos (*_tracked.mp4) are excluded from the repo due to file size — they are listed in .gitignore via the *.mp4 rule. To commit the videos too, comment out that line in .gitignore:

# *.mp4   ← comment this out to allow tracked output videos to be committed

Sequence: 3840×2160 · 24 fps · 280 frames Challenge: Uniform car roof + stationary road-marking corners competing with vehicle features Fix applied: Velocity-guided background filter

Tracking a silver car on a road viewed directly from overhead. With only ~66 corners available on the car's roof, the velocity filter is what keeps the bbox locked onto the vehicle rather than drifting to road markings.

Harris corners at initialisation and corner survival across 280 frames:

Point count across the full sequence — never drops below the 4-point minimum threshold (red dashed line). Mean: 48.0 corners/frame. Stepped drops are forward-backward filtering events; redetection fires at 25% survival when needed.

Centroid trajectory across all 280 frames (purple = early frames → yellow = late frames):

The trajectory maps the car's actual route: straight road section → right-angle intersection turn → continued travel. No position jumps, no drift.

📹 Watch Test 1 tracked output

Sequence: 3840×2160 · 30 fps · 529 frames Challenge: Vehicle exits frame with degrading point visibility Fix applied: Velocity prediction + exit-phase threshold

Tracking a vehicle across a rural road from an oblique angle. The perspective reveals the car's side panels, windows, and wheel arches — yielding 137 stable corners sustained across the entire 529-frame sequence without a single redetection event.

Point count held constant at 137 across every frame — no redetection triggered, no velocity fallback invoked until the exit phase. A clean, uninterrupted optical flow sequence from start to finish.

📹 Watch Test 2 tracked output

LK optical flow — frame-to-frame tracking (Test 1, frame 0→1):

All 66 corners tracked on the first inter-frame pass. Yellow dots show tracked feature locations; the green bbox shifts by the median displacement of surviving points.

Forward-backward error distribution — nearly all points cluster at ~0 px, well below the 3.5 px rejection threshold:

Periodic snapshots every 60 frames — bbox stays aligned through turns and background changes:

harris-lk-vehicle-tracker/
│
├── core/                              # Tracking algorithms — pure computer vision, no I/O
│   ├── harris_detector.py             # Harris corner detection with CLAHE + bbox mask
│   ├── lucas_kanade.py                # Pyramidal LK with forward-backward verification
│   └── object_tracker.py             # Stateful tracker: velocity, redetection, bbox update
│
├── utils/                             # Infrastructure utilities
│   ├── config_loader.py              # YAML loader with dot-access attribute interface
│   ├── folder_manager.py             # Timestamped output dir manager (runs never overwrite)
│   ├── health_check.py               # Pre-flight environment + dependency validation
│   └── resolution_utils.py           # Frame resolution helpers
│
├── metrics/                           # Observability
│   ├── logger.py                     # Levelled logging — INFO / DEBUG / WARNING
│   └── performance_tracker.py        # Per-frame metric accumulation and JSON export
│
├── visualization/                     # Output rendering — no tracking logic here
│   ├── display.py                    # BBox, centroid trail, and corner overlay rendering
│   └── plotter.py                    # Diagnostic plot generation
│
├── video_io/                          # I/O abstraction layer
│   ├── output_writer.py              # OpenCV VideoWriter wrapper (H.264 MP4)
│   └── video_source.py              # Video source with buffering
│
├── notebooks/                         # Phase-by-phase development notebooks
│   ├── phase1_infrastructure.ipynb   # Config loading, logging, folder management
│   ├── phase2_algorithms.ipynb       # Harris detection + cv2.selectROI bbox selection
│   ├── phase3_tracking.ipynb         # LK optical flow + forward-backward filtering
│   ├── phase4_results.ipynb          # Evaluation metrics, plots, analysis
│   └── phase5_integration.ipynb      # Full end-to-end pipeline — primary entry point
│
├── video/                             # Place test videos here (excluded via .gitignore)
│   └── README.md                     # Describes original test sequences
│
├── results/                           # Auto-generated on every run — never overwritten
│   └── test_N_YYYYMMDD_HHMMSS/
│       ├── videos/*_tracked.mp4      # ⚠ Excluded from repo (*.mp4 in .gitignore)
│       ├── logs/run.log
│       ├── plots/                    # Harris response, trajectory, survival, snapshots
│       ├── metrics/*.json
│       └── params/*.json             # Full config snapshot for reproducibility
│
├── report/
│   └── cv_project_tracking_report.pdf
│
├── main.py                            # CLI entry point
├── config.yaml                        # All tunable parameters
├── requirements.txt
├── CONTRIBUTING.md
└── LICENSE

main.py
  └── utils/config_loader.py
  └── utils/folder_manager.py
  └── utils/health_check.py
  └── video_io/video_source.py
  └── core/object_tracker.py
        ├── core/harris_detector.py
        └── core/lucas_kanade.py
  └── metrics/logger.py
  └── metrics/performance_tracker.py
  └── visualization/display.py
  └── visualization/plotter.py
  └── video_io/output_writer.py

No circular dependencies. core/ has no imports from visualization/, metrics/, or video_io/ — it is pure algorithm code and can be tested in isolation.

• Python 3.12
• Windows 11 (primary supported platform — see Troubleshooting for Linux/macOS notes)
• Jupyter Notebook or JupyterLab

1. Clone the repository

git clone https://github.com/whozahm3d/harris-lk-vehicle-tracker.git
cd harris-lk-vehicle-tracker

2. Create and activate a virtual environment

python -m venv venv

# Windows
venv\Scripts\activate

# macOS / Linux
source venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

4. Download the OpenH264 DLL (Windows only)

Required for H.264 MP4 output via OpenCV on Windows. Download openh264-1.8.0-win64.dll from the Cisco OpenH264 releases page and place it in the project root alongside main.py.

If the DLL is already present in the project root after cloning, skip this step.

5. Add your test video

Place your video file inside the video/ folder. The folder exists with a README.md describing the original test sequences — the actual video files are excluded due to size.

video/
└── your_video.mp4

6. Verify installation

python -c "import cv2, numpy, yaml; print('All dependencies OK')"

Only three fields change between videos. Everything else uses the defaults.

input:
  source: "C:\\path\\to\\your\\video\\test_1.mp4"   # Full path to your video file
  start_frame: 0                                      # Frame index to begin tracking from
  bbox: [x, y, width, height]                         # Bounding box in full-resolution pixels

Windows path note: Use double backslashes (\\) or forward slashes (/). Single backslashes will fail silently.

Open and run notebooks/phase2_algorithms.ipynb. It will:

• Open a frame scrubber to navigate the video and select the exact start frame
• Launch cv2.selectROI to draw a bounding box around the target vehicle
• Print the exact start_frame and bbox values to paste into config.yaml

Open and run notebooks/phase5_integration.ipynb. It will:

• Load config, initialise the tracker on the selected frame, and process all subsequent frames
• Display a live 1280×720 preview window during tracking (press Q to stop early)
• Save the annotated output video, logs, metrics, parameter snapshot, and all diagnostic plots to a timestamped folder under results/

python main.py --config config.yaml

Each run produces a self-contained, timestamped folder. Successive runs never overwrite each other.

results/
└── test_1_20260516_173936/
    ├── videos/
    │   └── test_1_tracked.mp4              # Annotated H.264 output video
    ├── logs/
    │   └── run.log                         # Full run log with per-frame point counts
    ├── plots/
    │   ├── integration_snapshots.png       # 6-panel tracking summary across the sequence
    │   ├── phase_2/
    │   │   ├── first_frame.png
    │   │   ├── harris_corners.png
    │   │   ├── harris_response.png
    │   │   └── quality_level_comparison.png
    │   ├── phase_3/
    │   │   ├── lk_tracking.png
    │   │   ├── fb_error_distribution.png
    │   │   └── bbox_update.png
    │   └── phase_4/
    │       ├── centroid_trajectory.png
    │       ├── point_survival.png
    │       ├── snapshots.png
    │       └── tracking_summary.txt        # Human-readable performance summary
    ├── metrics/
    │   └── test_1_metrics.json             # Avg / min / max points, total frame count
    └── params/
        └── test_1_params.json              # Full config snapshot for reproducibility

The following extensions are planned or under consideration. Contributions in these directions are welcome — see Developer Guide for how to propose and implement one.

• Scale-adaptive bounding box — estimate scale change per frame from the variance of tracked point displacements; resize bbox proportionally to handle zoom or perspective change
• Rotation-aware bbox — fit a minimum-area rotated rectangle to surviving points rather than an axis-aligned box; critical for vehicles making sharp turns at high frame intervals
• GPU-accelerated LK — replace cv2.calcOpticalFlowPyrLK with cv2.cuda.SparsePyrLKOpticalFlow; expected 10–20× throughput improvement on CUDA-capable hardware, enabling near-real-time 4K processing

• Automatic initialisation — integrate a lightweight detector (e.g. YOLOv8-nano or background subtraction) to eliminate the manual bbox requirement; tracker would remain classical post-initialisation
• Multi-object support — instantiate one ObjectTracker per detected vehicle with an ID-assignment layer; the current architecture already supports this — object_id is a first-class field in ObjectTracker
• RAFT/traditional flow comparison — benchmark against RAFT optical flow to quantify the accuracy gap and characterise the failure modes of the LK approximation

• Occlusion-aware redetection — use a learned re-identification embedding (e.g. OSNet) to verify that a redetected point cloud corresponds to the same vehicle, not a newly entered vehicle that happens to overlap the predicted bbox position
• Kalman filter integration (optional) — add an optional Kalman filter layer on top of the existing velocity EMA for smoother trajectory estimation; gated so the current purely-classical mode is preserved by default
• Benchmark on MOT-standard datasets — evaluate on UA-DETRAC or VisDrone sequences using standard HOTA/MOTA metrics; current validation is on custom sequences only

Branch naming:

feature/description-of-change     # new capability
fix/description-of-bug            # bug fix
refactor/description              # internal restructuring, no user-visible change
docs/description                  # documentation only

Commit style (imperative, present tense):

Add scale-adaptive bbox resizing
Fix velocity EMA not resetting on redetection
Refactor LK tracker to accept grayscale-only input

• Python 3.12, formatted with black (line length 100)
• Type hints on all public method signatures
• Docstrings on all public classes and methods (NumPy style)
• No function longer than 60 lines; extract helpers rather than nesting
• core/ must remain importable without video_io/, visualization/, or metrics/ — preserve the dependency direction

1. Open an issue first. Describe the problem, your proposed approach, and the expected impact. This avoids duplicate work and keeps the scope aligned with the project's classical CV focus.

2. Create a feature branch from main:

   git checkout -b feature/your-feature-name

3. Write the code. If your change touches core/, add a notebook under notebooks/ demonstrating the algorithm in isolation before integrating it into the pipeline.

4. Test on both sequences. Run the full pipeline on Test 1 and Test 2. Report tracker stability, avg points/frame, and processing rate in your PR.

5. Update config.yaml and the Configuration Reference if your feature introduces new parameters.

6. Open a pull request against main. Include:

   • A description of what changed and why
   • Before/after metrics from at least one test sequence
   • Any new plots or visual outputs in results/

This project is intentionally classical computer vision only. The following will not be merged:

• Deep learning models (YOLO, ResNet, transformers, etc.)
• PyTorch or TensorFlow dependencies
• Kalman filter implementations (unless gated behind a config flag that preserves the current default)
• Changes that break Python 3.12 compatibility

# Verify the full pipeline runs on the included config
python main.py --config config.yaml

# Check all imports resolve correctly
python -c "from core.harris_detector import HarrisDetector; print('core OK')"
python -c "from core.lucas_kanade import LKTracker; print('lk OK')"
python -c "from core.object_tracker import ObjectTracker; print('tracker OK')"

Failed to initialize VideoWriter or blank/corrupt output video

The OpenH264 DLL is missing from the project root.

Download openh264-1.8.0-win64.dll from:
https://github.com/cisco/openh264/releases

Place it in the project root alongside main.py.

Cannot open video error on launch

The path in config.yaml is wrong or the file does not exist at that location.

# Correct — double backslashes on Windows
source: "C:\\Users\\yourname\\harris-lk-vehicle-tracker\\video\\test_1.mp4"

# Also correct — forward slashes work on Windows
source: "C:/Users/yourname/harris-lk-vehicle-tracker/video/test_1.mp4"

# Wrong — single backslashes will silently fail
source: "C:\Users\yourname\video\test_1.mp4"

AttributeError: module 'ctypes' has no attribute 'windll'

You are running on Linux or macOS. The DPI awareness call in main.py is Windows-only and is already wrapped in a try/except block — this error should not surface in normal usage. If it does, verify you are running Python 3.12 and have not modified main.py.

Bounding box drifts off the vehicle during a sharp turn

The bbox is axis-aligned and fixed in size. During a sharp turn, the vehicle's silhouette rotates out of alignment, degrading displacement estimation. To improve: lower harris.quality_level from 0.005 to 0.003 to detect more corners on the vehicle body, and increase harris.max_corners from 400 to 600 for denser initial coverage. Drift will self-correct on straight sections as the point cloud re-stabilises.

Tracker deactivates before the vehicle exits frame

Increase memory.max_inactive_frames in config.yaml:

memory:
  max_inactive_frames: 90   # ~3 seconds at 30 fps

Processing is too slow

Expected — the system processes 4K footage at ~1.3–1.6 fps on CPU. To increase throughput: downscale the input video before running (e.g. to 1920×1080), or reduce lucas_kanade.max_level from 5 to 3 to cut pyramid computation at the cost of handling smaller maximum inter-frame displacements.

• No scale estimation — bounding box dimensions are fixed at initialisation. Objects changing apparent size due to camera zoom or perspective shift will have a progressively misaligned bbox.
• Single-object only — one ObjectTracker instance per run. Multi-vehicle tracking requires architectural changes beyond the current design.
• Manual initialisation required — a human must draw the bounding box on the start frame. No automatic detection stage exists.
• CPU-only throughput — ~1.3–1.6 fps on 4K input. Not suitable for real-time deployment without resolution downscaling or GPU-accelerated optical flow (e.g. cv2.cuda.SparsePyrLKOpticalFlow).
• Fixed occlusion budget — velocity prediction sustains tracking for up to 45 frames (~1.5 s at 30 fps). Longer full occlusions cause tracker deactivation.
• Axis-aligned bounding box — no rotation support. Vehicles turning sharply will have bbox misalignment proportional to the turn angle.
• Windows-primary — tested and validated on Windows 11 only. H.264 codec availability on Linux/macOS may differ.

Developed as the final project for the Fundamentals of Computer Vision course at FAST-NUCES Lahore. The full technical report covering algorithm design rationale, parameter analysis, and per-test evaluation is available in report/cv_project_tracking_report.pdf.

If you use this project in your research or coursework, please cite it as:

@misc{ahmad2026harrislk,
  author       = {Ahmad, Ali},
  title        = {Harris--LK Vehicle Tracker: Feature-Based Single-Object Tracking
                  using Harris Corners and Lucas--Kanade Optical Flow},
  year         = {2026},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/whozahm3d/harris-lk-vehicle-tracker}}
}

Contributions are welcome in the form of bug reports, parameter tuning suggestions, platform compatibility fixes, and documentation improvements. Please read CONTRIBUTING.md and the Developer Guide above before submitting a pull request. This project is intentionally classical CV only — no deep learning dependencies will be accepted.

This project is licensed under the MIT License. See LICENSE for full details.

Ali Ahmad · FAST-NUCES Lahore · Spring 2026 · github.com/whozahm3d