Moving away from OSF

[nichollsh]

We currently use OSF to host data files: spectral files, thermodynamics, etc. This is already somewhat problematic, as requests to OSF sometimes fail server-side. Additionally, OSF provides no manifest information and the API is rather opaque, which makes it hard to work out what data the code 'expects' to download.

Furthermore, the current climate in the US makes scientific data and other websites vulnerable. OSF is a US-based website.

I am wondering if we should find an alternative (and have two providers) or a replacement for OSF.

@timlichtenberg @lsoucasse thoughts?

[timlichtenberg]

The only website that I can really think of is https://zenodo.org/. Even AAS publishes their journal data sets there now: https://zenodo.org/communities/aas/records?q=&l=list&p=1&s=10&sort=newest. Zenodo is funded by the EU through CERN, therefore ultimately it will also be subject to political uncertainties. Nevertheless, I agree that it may be somewhat more secure in the mid-term.

We could move mid-term to a Zenodo communities, where we can again create an arbitrary amount of sub-repositories/data storages:
https://zenodo.org/communities/forming_worlds
https://zenodo.org/communities/proteus_framework

Would that work for you, Harrison? I would somewhat prefer to use the first community, but we can also use the second one. Once we have settled on this, we can edit the settings etc accordingly.

[lsoucasse]

I think Zenodo is a good option. We can set is as a backup or as the default.
I have spotted a python package to get files from there.
https://github.com/dvolgyes/zenodo_get

[nichollsh]

This seems promising. Is it possible to modify the files on Zenodo, such as when we update lookup tables, etc?

[timlichtenberg]

No, it's not, but you can update a repository to a new version v1, v2, v3 etc with updated files. As far as I understand the repo always defaults to the newest version. How this works in practice would need to be tested.