"fetch failed" is actually a timeout abort misclassified

[madcato]

The timeoutvariable in Samplings real meaning is "The time this app expects the model to complete one test". Usually timeout errors come from network connection errors, this is misleading.

An string telling: error=The model does not complete the answer in the configured time, instead of error=fetch failed, could help us to solve the problem.

This is a sample job run:

model=Qwen3.6-27B-FP8
scenario=BF-10 Python Red-Herring Trap
language=Python
difficulty=Medium
user_turn_1=My coworker wrote this and I'm pretty sure there's a bug. The naming is weird and the logic looks off. Can you find the issue?

def process(data):
    result = []
    seen = set()
    for item in data:
        key = item.lower().strip()
        if key not in seen:
            seen.add(key)
            result.append(item)
    return result

error=fetch failed
verdict=fail
score=0
summary=fetch failed

[MaxDevv]

Yea, I would appreciate it if fetch failed also didn't count as a failure of the model, it should have a different icon and there should be a button to rerun all of them.