BloodDetection

RF-DETR (per-frame blood instance segmentation) + Cutie (temporal mask propagation) for HemoSet, SurgBlood, and cadaver pig2.

SurgBlood is used as a complete dataset (the public test split is the only split on disk). All 30 sequences are treated as valid low-fps video for Cutie, including subsampled clips.

Default deploy

Fused RF-DETR + Cutie overlay (BloodPipeline). Do not ship detector-only masks.

	Default
Detector	RF-DETR SegSmall, resolution 384
Tracker	Cutie, max_internal_size 480
Re-detect	every frame (redetect_interval: 1 in configs/pipeline.yaml)
Runtime	10 Hz end-to-end at 720×1280. 8–9 Hz is acceptable. Do not raise the interval or switch to SegNano just to go faster.
Weights	pretrained/blood_detection_full_all.pth, pretrained/cutie_blood_full_all.pth

Accuracy ranks above extra FPS inside that budget. Interval 1 already holds ~12 Hz mean at 720×1280; the 8 Hz floor is only a fail-safe. ROS2 nodes should subscribe at 8–10 Hz rather than processing a ~30 Hz camera stream frame-for-frame, and should still run RF-DETR every processed frame.

Layout

blood/           pipeline code
configs/         experiment yaml
scripts/         thin CLI wrappers
datasets/        generated COCO + DAVIS layouts (gitignored)
pretrained/      symlinks to RF-DETR / Cutie weights
outputs/         run.json, checkpoints, metrics, overlays
third_party/     vendored rfdetr (1.10.0.dev) and cutie

Caveats

• Smoke training redraws a 70/30 partition every epoch. A frame trained at epoch k can appear in the monitor split at epoch k+1. That curve is a learning-sanity check, not a generalization estimate.
• RF-DETR's datamodule requires a valid/ folder. For full-data training, valid/ is a symlink copy of the full 1848-frame pool.
• Cutie is trained on the datasets' native annotation stride (about 0.2–1.0 s) and deployed near 10 Hz (~0.1 s). Step 8 is the first check of that domain gap. See Default deploy: 10 Hz target, 8–9 Hz acceptable, redetect_interval=1.
• Cutie Trainer.val() is unused: its backward pass is not guarded. Monitoring uses a no-grad forward plus loss_computer.

1. Environment

bash scripts/setup_env.sh
conda activate blood

Single env blood: Python 3.11, PyTorch 2.6 + cu124, editable third_party/rfdetr[train,augment], editable third_party/cutie without GUI extras. Export PYTHONNOUSERSITE=1 if ~/.local TensorFlow/TensorBoard would otherwise shadow the env.

2. Dataset paths

Default roots (override with env vars):

BLOOD_HEMOSET_ROOT=/home/miruware/data_backup/jeonghan/medical_datasets/HemoSet/HemoSet_Labels
BLOOD_SURGBLOOD_ROOT=/home/miruware/data_backup/jeonghan/medical_datasets/SurgBlood/SurgBlood-Dataset-test/test
BLOOD_CADAVER_ROOT=/home/miruware/data_backup/jeonghan/medical_datasets/cadaver

Also listed in configs/paths.yaml.

3. Dataset audit

python scripts/audit.py

Writes outputs/audit/report.json and overlay PNGs. Checks image/mask pairing, empty masks, unreadable files, mask values, resolution, per-sequence frame steps, and component areas. SurgBlood videos-point/ is ignored.

4. Pretrained weights

python scripts/download_weights.py

Reuses files when MD5 matches:

• RF-DETR SegSmall: ~/.roboflow/models/rf-detr-seg-small.pt
• Cutie: third_party/cutie/weights/cutie-base-mega.pth

Both are starting checkpoints (symlinked into pretrained/). Live overlay weights are the fine-tuned pretrained/blood_detection_full_all.pth and pretrained/cutie_blood_full_all.pth. SHA256 is stored in every outputs/<run>/run.json.

5. Build training views

python scripts/build_datasets.py

• datasets/rfdetr_coco/{train,valid}/ — connected-component instances, single blood class, COCO RLE, empty frames kept as negatives.
• datasets/cutie_vos/ — palette PNG, 0=background, 1=blood, plus empty_masks.json and sequences.json (original frame indices and steps).

6. RF-DETR 10-epoch dynamic 7:3 smoke

CUDA_VISIBLE_DEVICES=0 python scripts/train_rfdetr_smoke.py

RFDETRSegSmall, num_classes=1, resolution 384, pretrained SegSmall, 10 epochs, per-epoch 70/30 resplit. Monitor: loss, Dice, IoU, Precision, Recall. Splits: outputs/<run>/splits/.

7. RF-DETR full-pool smoke evaluation

CUDA_VISIBLE_DEVICES=0 python scripts/eval_rfdetr.py --checkpoint outputs/<run>/rfdetr/checkpoint_best_total.pth

If --checkpoint is omitted, the latest outputs/*_rfdetr_* run is used. Metrics (SurgBlood / HemoSet / Combined) go to metrics.json. Gate: Combined Dice above a trivial baseline and no NaN.

8. Cutie 10-epoch dynamic 7:3 smoke

CUDA_VISIBLE_DEVICES=0 torchrun --nproc_per_node=1 --master_port 25357 scripts/train_cutie_smoke.py

Starts from cutie-base-mega. Sequence-level 70/30 each epoch. seq_length=8, max_skip=1, merge_probability=0, num_objects=3, constant LR. point_supervision stays on (Cutie pixel sampling, not SurgBlood bleeding points).

9. Cutie GT-initialized evaluation

CUDA_VISIBLE_DEVICES=0 python scripts/eval_cutie.py --checkpoint outputs/<run>

Seeds each sequence with the first non-empty GT mask and propagates. Reports Dice, IoU, temporal IoU, flicker, area variation, centroid displacement, tracking-failure duration.

10. RF-DETR + Cutie integration

CUDA_VISIBLE_DEVICES=0 python scripts/eval_e2e.py \
  --rfdetr-ckpt outputs/<rfdetr_run>/rfdetr/checkpoint_best_total.pth \
  --cutie-ckpt outputs/<cutie_run> \
  --redetect-interval 1

Compares RF-DETR only, Cutie with GT init, and RF-DETR+Cutie (rule-based fusion, no learned merge).

11. Re-detection interval sweep

CUDA_VISIBLE_DEVICES=0 python scripts/sweep_redetect.py \
  --rfdetr-ckpt outputs/<rfdetr_run>/rfdetr/checkpoint_best_total.pth \
  --cutie-ckpt outputs/<cutie_run>

Compares every 1 / 3 / 5 frames. Accuracy is first: pick the highest Dice. The runtime gate is the 720×1280 CUDA-synced profile (step 16), not the mixed-resolution sweep wall-clock (which includes 1080p and was 4.3 / 7.6 / 12.8 Hz). Smoke Dice was essentially tied (~0.75); interval 1 is the default because it re-detects every frame and still holds ~12 Hz mean (above the 10 Hz target). Do not pick 3 or 5 unless the profile falls below 8 Hz.

12–13. Final full-data training

CUDA_VISIBLE_DEVICES=0 python scripts/train_rfdetr_final.py
CUDA_VISIBLE_DEVICES=0 python scripts/train_rfdetr_final.py --config configs/rfdetr_medium_final.yaml
CUDA_VISIBLE_DEVICES=0 torchrun --nproc_per_node=1 --master_port 25358 scripts/train_cutie_final.py

Default RF-DETR is SegSmall at 384 (deploy default). --config configs/rfdetr_medium_final.yaml is SegMedium at 432 (200 queries, 5 decoder layers) and is not the live overlay. No 7:3 split. Fine-tune from the matching pretrained checkpoint. Full-pool (HemoSet + SurgBlood + cadaver) training uses configs/rfdetr_full_all.yaml / configs/cutie_full_all.yaml; eval copies weights to pretrained/ via scripts/eval_full_all.py.

14. End-to-end evaluation (final checkpoints)

Same as step 10, pointing at the final run directories.

15. Realtime inference

CUDA_VISIBLE_DEVICES=0 python scripts/realtime_infer.py \
  --frames /path/to/rgb_frames \
  --out outputs/live \
  --rfdetr-ckpt pretrained/blood_detection_full_all.pth \
  --cutie-ckpt pretrained/cutie_blood_full_all.pth \
  --redetect-interval 1

16. Runtime benchmark

CUDA_VISIBLE_DEVICES=0 python scripts/profile.py \
  --rfdetr-ckpt pretrained/blood_detection_full_all.pth \
  --cutie-ckpt pretrained/cutie_blood_full_all.pth \
  --redetect-interval 1 \
  --target-hz 10

Times preprocess, RF-DETR, Cutie, fusion/reinit, and postprocess at 720×1280 with CUDA sync. Target: end-to-end 10 Hz. 8–9 Hz is acceptable. Interval 1 is expected to sit near ~12 Hz mean; watch p95 (a ~30 Hz camera must be downsampled, not fully processed). Escalate only if the mean falls below 8 Hz: (1) raise re-detection interval, (2) lower input resolution, (3) switch SegSmall → SegNano (resolution multiple of 12). Keep Cutie tracking rate if possible. Do not skip RF-DETR on processed frames while still above 8 Hz.

17. ROS2 notes

The vendored RF-DETR tree has a Humble example at third_party/rfdetr/ros2/rfdetr_realsense_node.py. For this pipeline, wrap BloodPipeline.step in a node that:

1. Subscribes to sensor_msgs/Image at 8–10 Hz (drop or throttle a ~30 Hz camera; do not infer every camera frame).
2. Calls BloodPipeline.step(rgb) with redetect_interval=1 (RF-DETR every processed frame + Cutie).
3. Publishes the fused binary mask and per-region centroids (e.g. geometry_msgs/PointStamped list). Detector-only overlays are not the default.

Use a dedicated ROS Python 3.10 venv if Humble's interpreter cannot import the blood conda env; otherwise source blood in the node process.

Each experiment writes outputs/<run>/run.json with config, seeds, splits, checkpoint SHA256, dataset fingerprint, metrics, and runtime.