Video Annotation Tools

What This Is

A multi-tracker video annotation platform built for generating high-quality training data for computer vision models. The bottleneck in computer vision is rarely the model it's the labels. Annotation throughput and quality directly determine model quality. This tool is engineered to maximise both.

Why Annotation Quality Is the Real Problem

ImageNet took over 25,000 workers and years to annotate. SAM2, Meta's current SOTA video segmentation model, was trained on the SA-V dataset 51k videos with 643k masklets, requiring a custom annotation interface to achieve tractable throughput. The quality of the annotation pipeline is inseparable from the quality of the model.

For video specifically, the challenge is temporal consistency: bounding boxes must track the same object instance across frames, even through occlusion, appearance change, and rapid motion. Inconsistent annotations teach the model inconsistent behaviour.

The Tracker Suite

We integrate six OpenCV tracking algorithms with deliberately different characteristics, letting annotators choose the right tool for the scene:

• CSRT: highest accuracy, slower. Best for small or irregularly shaped objects.
• KCF: fast, good for predictable motion. Best for rigid objects moving smoothly.
• MIL: multiple instance learning, robust to partial occlusion.
• BOOSTING: oldest method, baseline.
• TLD: re-detects after tracking failure. Best for objects that temporarily leave frame.
• MEDIANFLOW: detects tracking failures and stops. Best for slow, predictable motion where failure detection matters.

Template matching recommended for very slow camera motion.

The Annotation Format

Output is JSON with frame-level bounding boxes, object IDs, and track confidence. The schema is compatible with COCO video annotation format, making it directly usable for fine-tuning YOLO, Detectron2, and other standard detectors.

What We're Building Toward

• Integration with SAM2 for zero-shot object segmentation: click once, get a mask for the full video
• Active learning mode: the annotation tool flags frames where the model is most uncertain, focusing human effort where it matters most
• Quality metrics: inter-annotator agreement scores for validation splits