Problem
evaluate.metrics.get_stats() crashes when a requested key has no valid (non-NaN) data across any sample in the dataset.
Failure 1: all-NaN labels
If every sample has NaN for a given key (e.g. stress is missing for the whole dataset but "stress" is in keys), tmp[key] remains an empty list. np.concatenate([]) then raises:
ValueError: need at least one array to concatenate
Failure 2: missing key
If a key is requested but doesn't exist in sample.labels at all (not NaN, just absent), the label lookup raises a KeyError.
Reproducer
playground/repro_get_stats_nan.py:
import numpy as np
from collections import namedtuple
from marathon.evaluate.metrics import get_stats
Sample = namedtuple("Sample", ("structure", "labels"))
samples = [
Sample(
{"positions": np.zeros((3, 3))},
{
"energy": 1.0,
"forces": np.ones((3, 3)),
"stress": float("nan") * np.ones((3, 3)),
"num_atoms": 3,
},
),
]
# Crashes — stress is all-NaN:
stats = get_stats(samples, keys=["energy", "forces", "stress"])
Where
marathon/evaluate/metrics.py, get_stats(), lines ~120–154.
Suggested fix
- Skip keys with no valid data: omit from returned stats or fill with NaN sentinels.
- Guard label lookup with
key in l check (treat missing same as NaN).
- Downstream (
get_metrics_fn, emit) needs to handle absent stats gracefully (skip R² when unavailable).
Problem
evaluate.metrics.get_stats()crashes when a requested key has no valid (non-NaN) data across any sample in the dataset.Failure 1: all-NaN labels
If every sample has NaN for a given key (e.g. stress is missing for the whole dataset but
"stress"is inkeys),tmp[key]remains an empty list.np.concatenate([])then raises:Failure 2: missing key
If a key is requested but doesn't exist in
sample.labelsat all (not NaN, just absent), the label lookup raises aKeyError.Reproducer
playground/repro_get_stats_nan.py:Where
marathon/evaluate/metrics.py,get_stats(), lines ~120–154.Suggested fix
key in lcheck (treat missing same as NaN).get_metrics_fn,emit) needs to handle absent stats gracefully (skip R² when unavailable).