Temporal-aware Multimodal Fusion Network (TMFN)

Code and data accompanying the manuscript:

Multimodal Deep Learning Model for Forecasting Using Textual and Numerical Signals

This repository hosts the reference implementation of TMFN, a BiLSTM + frozen-BERT forecaster with quantile heads and conformalised prediction intervals, applied to [climate / health] forecasting as described in the manuscript.

---

Status: held for release pending peer review

The full source code is not yet public. It will be released here under an open-source license, upon acceptance of the manuscript. In addition, the source code will be uploaded and attached with the revision material for review.

This staged release does not restrict evaluation of the work:

• Reviewers and editors may request the complete code and data for peer review at any time. Access will be provided promptly through the editorial office / a private repository.
• The method is described in full in the manuscript and its Supplementary Information, sufficient to understand and assess the approach.

Method summary

• Numeric branch: BiLSTM with additive attention over a sliding window of engineered features.
• Text branch: frozen BERT encoder over a concatenated window of contemporaneous text/search features.
• Fusion: a learned cross-gating mechanism combines the two modalities; ablations replace it with concatenation or drop the text branch entirely.
• Heads: a mean plus ordered quantiles (5/50/95), trained with a pinball + MSE objective, with split-conformal calibration of the 90% prediction interval.
• Evaluation: multi-seed ensembling, coverage / interval-width / Winkler score, and an ablation suite isolating the contribution of the text modality and the gate.

Full architecture, hyperparameters, and training details are in the manuscript Methods and Supplementary Information.

Data availability

The datasets used in this study are based on the Time-MMD benchmark datasets, available from: https://github.com/AdityaLab/Time-MMD/

The processed data used for the experiments will be deposited with the code upon acceptance, and the repository will also document the data preparation steps required to reproduce the analyses.

Planned public release (upon acceptance)

At acceptance this repository will contain:

• the installable tmfn package (model, training, evaluation, and the domain adapter(s) reported in the paper);
• exact commands and configurations to reproduce the reported results;
• an environment specification (pyproject.toml / requirements.txt);