Add Transformers 5.x Compatibility

[kylehowells]

Summary

This PR adds support for transformers>=5.0.0 while maintaining full backward compatibility with transformers 4.x. The fix addresses a breaking change in how HuggingFace transformer models return outputs when add_pooling_layer=False.

Problem

With transformers==5.0.0, COMET fails with the following error when scoring translations:

ValueError: not enough values to unpack (expected 3, got 2)

This occurs in comet/encoders/xlmr.py:95 and comet/encoders/bert.py:171 where the code expects 3 values from the model output tuple.

Root Cause

In transformers 5.0, when models are created with add_pooling_layer=False, the output tuple format changed:

Version	Output Format
Transformers 4.x	(last_hidden_states, pooler_output_or_None, hidden_states_tuple)
Transformers 5.x	(last_hidden_states, hidden_states_tuple)

The pooler output is now omitted entirely rather than being included as None.

Solution

Updated the encoder forward() methods to dynamically handle both output formats by checking the tuple length:

output = self.model(...)
if len(output) == 2:
    last_hidden_states, all_layers = output
else:
    last_hidden_states, _, all_layers = output

Changes

• comet/encoders/xlmr.py: Updated XLMREncoder.forward() to handle both 2-value and 3-value output tuples
• comet/encoders/bert.py: Updated BERTEncoder.forward() with the same fix
• pyproject.toml: Relaxed version constraints:
  • transformers = "^4.17" → transformers = ">=4.17"
  • huggingface-hub = ">=0.19.3,<1.0" → huggingface-hub = ">=0.19.3"

Testing

Tested with Unbabel/wmt22-comet-da model on sample translation pairs:

Transformers Version	Status	System Score	Sentence Scores
4.57.3	Pass	0.9316	[0.9714, 0.8917]
5.0.0	Pass	0.9316	[0.9714, 0.8917]

Scores are identical, confirming backward compatibility and correctness.

---

The Transformers 5 release broke some of my scripts using Comet to benchmark translation scores due to the return value being different now, so worked on this PR (with Claude Code and Opus 4.5) to fix it.

I ran the translation scores on Spanish and checked everything still matched before and after. I've read and looked over the change and it looks ok to me, fairly minimal change that doesn't seem very disruptive to the code base.