YOLO and Reprojection Filtering Update

docs/Writerside/topics/resources/reprojection_filtering.md

@@ -1,21 +1,21 @@
-# Reprojection Error Filtering
+# Reprojection Error Outlier Rejection
 
-Reprojection error filtering is an optional postprocessing step to the 3D triangulation stage of processing. It retriangulates outlier data from the 3d triangulation with the cameras contributing the most error removed. Reprojection error filtering is most effective when there is poor skeleton detection in one or more camera views. It is turned on and off in "Process Data" tab with the checkbox "Run Reprojection Error Filtering". It is turned off by default.
+Reprojection error outlier rejection is an optional postprocessing step to the 3D triangulation stage of processing. It retriangulates outlier data from the 3d triangulation with the cameras contributing the most error removed. Reprojection error outlier rejection is most effective when there is poor skeleton detection in one or more camera views. It is turned on and off in "Process Data" tab with the checkbox "Use Outlier Rejection Method?". It is turned off by default.
 
-![Detail of Reprojection Error Filtering Options](reprojection_filtering_crop_detail.png){ width="450" }
+![Detail of Reprojection Error Outlier Rejection Options](reprojection_filtering_crop_detail.png){ width="450" }
 
 ## What is Reprojection Error?
 "Reprojection error" is the distance between the originally measured point (i.e. a joint on the 2d skeleton) and the reconstructed 3d point reprojected back onto the original image. The intuition is that if the 3d reconstruction and original 2d track are perfect, then reprojection error will be zero. If it isn't, then there is some inaccuracy in either: the original 2d tracks (i.e. bad skeleton detection from one or more cameras), in the 3d reconstruction (i.e. bad camera calibration), or a combination of the two.
 
-## How Does Reprojection Error Filtering Work?
-Reprojection error filtering can help when there is poor skeleton detection in one or more camera views. It starts by finding the outliers in the reprojection error, defined as all points above a certain threshold error value calculated as a percentile of the data. It then retriangulates the data with the camera contributing the most error (for that specific frame and marker) removed. If there are still reprojection errors above the threshold, it repeats the process but also removes the camera that contributes the second most to the error. It continues this process until all reprojection errors are below the threshold, or until it hits the minimum number of cameras left. If it hits the minimum camera limit, it sets the remaining data that still has a reprojection error above the threshold to NaNs, which will then be gap filled in the post processing stage. Once this process is complete, it reinserts the filtered data back into the normal data.
+## How Does Reprojection Error Outlier Rejection Work?
+Reprojection error outlier rejection can help when there is poor skeleton detection in one or more camera views. After running the standard triangulation on a marker with all cameras, it calculates the reprojection error. If the error is above the target threshold, it iteratively tests subsets of cameras by dropping on or more cameras at a time, up to a configurable maximum. During this process, it tests each combination of cameras and calculates an exponential weight based on the reprojection error for that subset. If any combination within one loop achieves an error below the target threshold, the search stops early to prevent dropping more cameras than necessary. The result is the weighted average 3D point, with points that have lower reprojection error being weighted higher. If no subset of cameras achieves an error below the target, then the returned value is the one with the lowest error between the weighted result and the initial triangulation.
+
 
 ## What are the Parameters and What Do They Do?
-The first parameter is "Run Reprojection Error Filtering", a checkbox that sets whether to run the reprojection error filtering process. It is turned off by default.
+The first parameter is "Use Outlier Rejection Method?", a checkbox that sets whether to run the reprojection error outlier rejection process. It is turned off by default.
 
-The second parameter is "Reprojection Error Threshold (%)", which sets the percentile above which reprojection errors will be filtered out. The percentile threshold will be calculated to an absolute error, and then that absolute error will be the comparison for the data. Setting a lower threshold will result in more data being retriangulated, while setting a higher threshold will result in less data being retriangulated. It is set to 90% by default.
+The second parameter is "Maximum Cameras to Drop", which sets the maximum number of cameras that can be removed during the search. The default is set to 1. 
 
-The third parameter is "Minimum Number of Cameras Left", which sets the minimum number of cameras to remain in the triangulation process during the reprojection error filtering. If the filtering process is able to remove all outliers before the minimum number of cameras is reached, the reprojection error filtering process will stop. If outliers remain after the minimum number of cameras is reached, the remaining outlier data will be set to NaNs. The minimum possible value is 2, as two camera views are required for 3D triangulation. If the minimum number of cameras is greater than or equal to the total number of cameras in the recording, the reprojection error filtering process will not run. It is set to 3 by default.
+The third parameter is "Minimum Cameras for Triangulation", which sets the minimum number of cameras. If you are running 4 or more cameras, you may want to increase this number. The number of cameras used will be limited by both this parameter and the "Maximum Cameras to Drop". The default is set to 2.
 
-## Note about Debug Plots
-A debug plot for the reprojection error filtering is saved to the `output_data/raw_data` folder of your recording session. The reprojection error filtering runs on a per camera and per marker level, while the debug plot displays the mean reprojection error for a frame across all body markers and cameras. This can lead to plots that don't show the filtering results very well.
+The fourth parameter is "Target Reprojection Error", which sets the mean reprojection error to aim for when filtering each marker. Lowering this will lead to rejecting more outliers, and raising it will lead to accepting more outliers.

docs/Writerside/topics/resources/yolo_cropping.md

@@ -1,6 +1,6 @@
 # YOLO Cropping
 
-YOLO cropping is a processing option that passes each video frame through YOLO v8's object detection model to find the most likely area of the person in the frame before passing the image to the pose estimation model. This can help reduce false positives during pose estimation and aid tracking subjects that are relatively small in the cameras field of view. This process increases the quality of the output data at the cost of increased processing time. It can be turned on and off in "Process Data" tab with the checkbox "Use YOLO Crop Method". It is turned off by default.
+YOLO cropping is a processing option for recording situations where the subject is relatively small inside the image. It passes each video frame through YOLO v8's object detection model to find the most likely area of the person in the frame, and then passes this cropped region of interest to the pose estimation model. This can help reduce false positives during pose estimation and aid tracking subjects that are relatively small in the cameras field of view. This increases processing time, and may reduce processing quality for standard recordings. It should only be used when cameras need to be placed far from a subject. It can be turned on and off in "Process Data" tab with the checkbox "Use YOLO Crop Method". It is turned off by default.
 
 ![Detail of YOLO Crop Checkbox](YOLO_crop_detail.png){ width="450" }
 
@@ -9,4 +9,4 @@ The pose estimation models FreeMoCap uses are all set up to process a certain si
 
 YOLO cropping is a preprocessing stage that runs before the pose estimation. Instead of sending the entire video frame into the pose estimation model, it first runs an object detection model that looks for people. Unlike a pose estimation model that tracks joint locations, the output of the object detection model is a bounding box that shows where the person in the image is. We then crop the image based on the bounding box and send the cropped image into the pose estimation model. This means we start the downsampling with the most relevant information, and are able to keep as much information as possible when running the pose estimation.
 
-YOLO cropping is beneficial in most recording situations. It will be most helpful when the subject is small in a camera's field of view, or in situations where the pose estimation is returning a lot of false positives (your annotated videos will have skeletons drawn not on a person). You may choose to skip YOLO cropping when processing time is more important than tracking quality, or when the subject already fills most of the camera views. YOLO cropping roughly doubles the processing time of the 2D Image Tracking stage of the pipeline.
+YOLO cropping is beneficial when cameras are far from the subject or very wide angle. It will be most helpful when the subject is small in a camera's field of view, or in situations where the pose estimation is returning a lot of false positives (your annotated videos will have skeletons drawn not on a person). YOLO cropping may negatively impact tracking quality when the subject already fills most of the camera views. YOLO cropping roughly doubles the processing time of the 2D Image Tracking stage of the pipeline.