DOTA

[YXB-NKU]

how to get calibrators for rotated object detector ,can you provide more details

[YXB-NKU]

I want to test the results of an ensemble of two other models. However, when I read the .pkl file, I found that after calibration, the y_max for all 15 categories was 1, and the y_min was 0. To proceed, I simply replaced the folder names and swapped the test results in work_dirs with those from the other two models. Then, I used the mocae_rotated_object_detection.py script to calibrate the test results of these two models. However, I found that the performance was much worse than the original results of the two models.

[kemaloksuz]

We haven't released the calibration code for rotated object detection yet. We are planning to release that part as well but it is very unlikely that it will be released before ECCV. So, it can take between 1-2 months before we release it.

If you do not want to wait till then, you can also implement by modifying model_calibration.py accordingly (this is only supporting object detection at the moment). Overall, (1) it should read the detections in the validation set (you can see the current code to understand the format etc.) and (2) make the assignment between detections and gts based on IoUs. The second part can be implemented by using box_iou_quadri (from mmcv.ops import box_iou_quadri) and then replacing assign_post function with the function below:

def assign_rotated(ann_dict, det_bboxes, det_score, det_label, dataset_classes):
    num_classes = len(dataset_classes)
    ious = np.zeros([det_bboxes.shape[0]])
    ## Assign
    for k, v in ann_dict.items():
        # Convert to numpy and reshape
        gt_boxes = np.array(v).reshape(-1, 8)
        rel_idx = (det_label==k).nonzero()[0]
        ious_cl = (box_iou_quadri(torch.from_numpy(gt_boxes).float(), torch.from_numpy(det_bboxes[rel_idx]).float())).numpy()
        ious[rel_idx] = np.max(ious_cl, axis=0)
    return ious

Hope this helps, and sorry for delaying the calibration part for that task.

[kemaloksuz]

Hi @rookie0607, thanks for your interest.

We have an intent to do that as I mentioned above. However, our schedule is quite busy and we could not have chance to look into this yet. We will let you know once we could incorporate this part into the code.

[Ideal-111]

@kemaloksuz Hi, I want to know how to deal with the DOTA dataset when I modify model_calibration.py script.

[Ideal-111]

@kemaloksuz Specifically, I used the oriented R-CNN model(use dota-v1.0 dataset) and got a .pkl format file and .txt format files from mmrotate, but how should I convert these files to coco-style appropriately, can you give me some advice?

[kemaloksuz]

@kemaloksuz Hi, I want to know how to deal with the DOTA dataset when I modify model_calibration.py script.

The idea is similar to standard object detection. You just need to compute IoU between the rotated bounding boxes instead of the IoU between the rectangles, and use it as the calibration target. The code snippet that I had provided above was actually doing that. (I was importing box_iou_quadri from mmcv most likely but not perfectly sure.)

In the most recent zip file, I also provided the detections for both detectors on the validation set. So, you can try to reproduce. To summarize:

• Train the calibrator using the detections on the validation set by replacing iou by rotated_iou
• Just replace the calibrator by the calibrator you trained in the inference script I provided.

Please let me know if you do it and reproduce, and we'd apreciate a PR on this as unfortunately we cannot prioritize adding this feature at the moment.

[Ideal-111]

@kemaloksuz I obtained the same calibrators and AP results after modifying the model_calibration.py.

[kemaloksuz]

@Ideal-111 that is great to hear. Thanks for letting me know.

Would you like to open a pull request for it? It could be quite useful as there has been some interest for calibrating rotated object detectors. I can then review and merge it.

[Ideal-111]

@kemaloksuz I've opened.