Skip to content

Missing support for ground truth files where no events occur #12

@benjaminsshoffman

Description

@benjaminsshoffman

If a ground-truth file does not contain any events, we get an error ValueError: ground_truth audio ids do not match audio_ids.

psds1, percls_psds1, _, _ = sed_scores_eval.intersection_based.psds(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/home/representation-learning/.venv/lib/python3.12/site-packages/sed_scores_eval/intersection_based/psds.py", line 89, in psds
    (effective_tp_rate, effective_fp_rate), single_class_psd_rocs = psd_roc(
                                                                    ^^^^^^^^
  File "/mnt/home/representation-learning/.venv/lib/python3.12/site-packages/sed_scores_eval/intersection_based/psds.py", line 192, in psd_roc
    intermediate_statistics, audio_ids = accumulated_intermediate_statistics(
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/home/representation-learning/.venv/lib/python3.12/site-packages/sed_scores_eval/intersection_based/intermediate_statistics.py", line 168, in accumulated_intermediate_statistics
    scores, ground_truth, audio_ids = parse_inputs(scores, ground_truth)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/home/representation-learning/.venv/lib/python3.12/site-packages/sed_scores_eval/base_modules/io.py", line 35, in parse_inputs
    ground_truth = parse_ground_truth(ground_truth, tagging=tagging, audio_ids=audio_ids)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/home/representation-learning/.venv/lib/python3.12/site-packages/sed_scores_eval/base_modules/io.py", line 69, in parse_ground_truth
    raise ValueError(

There is a way to pass audio_ids in base_modules.io.parse_ground_truth, but this option isn't accessible from the high-level functions I want to call (e.g. psds), as far as I can tell.

This is worth fixing because in detection settings, sometimes we have files where no events occur, but we still want to avoid false positives in these cases.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions