-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
User Story
As a video analysis user,
I want multiple frame comparison methods in extract_significant_frames
so that semantic changes between visually similar frames are captured more effectively.
Background
The current grayscale pixel difference method in VideoDescriber.extract_significant_frames (lines 33-49 of video_describer.py) only detects absolute pixel value changes. This fails to capture meaningful content changes when frames have similar brightness distributions but different color patterns or object movements (e.g., a red object turning blue while maintaining luminance). The existing threshold-based approach also lacks context-aware comparison capabilities.
Acceptance Criteria
- Modify
frame_story/video_describer.pyto implement:
• Histogram comparison using OpenCV'scv2.compareHistwith correlation method
• Dense optical flow analysis viacv2.calcOpticalFlowFarneback - Add
methodparameter toVideoDescriber.__init__with options:
["grayscale" (default), "histogram", "optical_flow"] - Update frame significance logic to:
- For histogram mode: Compare HSV histograms using 16-bin quantization
- For optical flow: Calculate magnitude thresholding on flow vectors
- Validate through unit tests in
tests/test_video_describer.py:- Ensure all methods detect frame changes in test videos with:
- Same composition but different color schemes (must fail with grayscale)
- Static background with moving foreground objects
- Maintain ≤5% false positive rate compared to current implementation
- Ensure all methods detect frame changes in test videos with:
- Keep backward compatibility by preserving default grayscale behavior
- Add configuration parameters for histogram/optical flow thresholds in
VideoDescriberinitialization
Reactions are currently unavailable