We propose a multi-horizon forecasting approach that accurately models the underlying patterns on different time scales. Our approach is based on the transformer architecture, which across a wide range of domains, has demonstrated significant improvements over other architectures. Several approaches focus on integrating a temporal context into the query-key similarity of the attention mechanism of transformers to further improve their forecasting quality. In this paper, we provide several extensions to this line of work. We propose an adjustable context-aware attention that dynamically learns the ideal temporal context length for each forecasting time point. This allows the model to seamlessly switch between different time scales as needed, hence providing users with a better forecasting model. Furthermore, we exploit redundancies arising from incorporating the temporal context into the attention mechanism to improve runtime and space complexity. Our experiments on several real-world datasets demonstrate significant performance improvements over existing state-of-the-art methodologies.
- Python 3.8.3
- torch 1.9.0
- Numpy 1.18.5
- Pandas 1.0.5
git clone https://github.com/SepKfr/Adjustable-context-aware-transfomer.git
bash run.sh
A containerized snapshot of the research code and reproducibility materials is available through the GitHub Container Registry:
docker pull ghcr.io/sepkfr/adjustable-context-aware-transfomer:v0.1.0