Skip to content

Conversation

@westlinkin
Copy link

Description

This PR introduces CameraMotionCompensator, a utility class that improves tracking performance with camera motion. This addresses the problem where camera movement (panning, shaking, etc.) degrades tracker accuracy, leading to ID switches and lost tracks.

No extra dependencies that are required for this change.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

The feature has been tested using a real-world video processing script. The following example demonstrates how to use the CameraMotionCompensator in "Simple Mode" within the sv.process_video callback structure. This confirms that the motion calculation is performed once per frame and correctly applied to the tracker.

import numpy as np
import cv2
import supervision as sv
from ultralytics import RTDETR

def process_video_with_cmc():
    # Setup models, video paths, etc.
    helmet_model = RTDETR(<HELMET_MODEL_PATH>)
    clip_path = <VIDEO_PATH>
    output_path = "result.mp4"

    # Initialize tools
    box_annotator = sv.BoxAnnotator()
    label_annotator = sv.LabelAnnotator()
    
    # Initialize tracker and wrap it with CameraMotionCompensator
    byte_tracker = sv.ByteTrack(frame_rate=<FRAME_RATE>)
    cmc = sv.CameraMotionCompensator(tracker=byte_tracker)
    cmc.reset()

    def callback(frame: np.ndarray, index: int) -> np.ndarray:
        # 1. Perform detection
        results = helmet_model(frame)[0]
        detections = sv.Detections.from_ultralytics(results)

        # 2. Update with detections using the simple wrapper method
        final_detections = cmc.update_with_detections(detections, frame=frame)

        # 3. Annotation
        labels = [
            f"#{tracker_id} {class_name} {confidence:.2f}"
            for class_name, tracker_id, confidence
            in zip(final_detections.data["class_name"], final_detections.tracker_id, final_detections.confidence)
        ]
        annotated_frame = box_annotator.annotate(frame.copy(), detections=final_detections)
        return label_annotator.annotate(annotated_frame, detections=final_detections, labels=labels)

    sv.process_video(
        source_path=clip_path,
        target_path=output_path,
        callback=callback
    )

if __name__ == "__main__":
    process_video_with_cmc()

Any specific deployment considerations

None

Docs

  • Docs updated? What were the changes:

@westlinkin westlinkin requested a review from SkalskiP as a code owner November 6, 2025 15:11
@westlinkin westlinkin changed the title Add CameraMotionCompensator to improve tracking performance with camera motion feat: add CameraMotionCompensator to improve tracking performance with camera motion Nov 6, 2025
@SkalskiP
Copy link
Collaborator

SkalskiP commented Nov 6, 2025

Hi @westlinkin 👋🏻 Thanks a lot for your interest in Supervision. Could you share a Google Colab notebook showing how CMC is used in practice? It would help me review this PR faster.

@westlinkin
Copy link
Author

I cannot share my custom trained model, but here is the results of without cmc video and cmc video:

No. 4 is missed in no-cmc version at the end of the video

Here is the colab notebook with a default YOLO model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants