About rollout fine-tuning implementation

[soberguo]

I am currently working on fine-tuning the Aurora model. While I have achieved satisfactory results with single-step predictions, I am encountering significant challenges with the performance of rollout (multi-step autoregressive) predictions.

I have reviewed the supplementary materials in detail, but I found the description regarding the specific implementation of the rollout mechanism to be somewhat limited. As a result, I am facing difficulties in reproducing stable long-term forecasts and optimizing the rollout fine-tuning process.

Could you please provide some additional guidance or reference materials? Specifically, I would appreciate:

Implementation Details: Any best practices or "tricks" for stabilizing rollout training.

Related Resources: Are there any related GitHub repositories, code snippets, or specific papers that discuss the autoregressive fine-tuning strategy used for Aurora in more depth?

Any insights or pointers to relevant resources would be greatly appreciated.

Thank you for your time and for this excellent work.