Offer to modernize PyDSD and take a lead role.

[scollis]

Good morning PyDSD Developers and contributors,
PyDSD is actively used in the Atmospheric Radiation Measurement (ARM) User Facility. As you may know PyTMatrix has become a challenge and is not supported. Recently @swnesbitt developed rustmatrix. We have confirmed this behaves very similarly to PyTMatrix and can be used as a drop in replacement. ARM is currently deciding if we want to start our own code similar to PyDSD. However, we really want the history of this project to be respected so we are also discussing if ARM can "take over" development leadership of PyDSD. Is this something that would be welcomed? We would move the base fork into https://github.com/ARM-DOE/ and this would maintain the commit history AND we would maintain your citation notice so you continue to get credit. Please let us know as soon as possible!

[scollis]

Hey all. Nothing heard. I'll consult with ARM. But please do reach out. We do want to acknowledge the hard work put in by this dev team

[nguy]

@scollis Sorry for the delay. I'm not the primary developer here, but this repository has not been maintained at the level you are suggesting and I for one would rather see it used in development in some way moving forward. So I support ARM taking over this repository development from here!

[scollis]

Thanks @nguy. We'll be discussing this and moving on it soon. What we may do is start a a new repo and nod to the history of PyDSD. We want to also modernize the data models etc (xarray, datatree)

[swnesbitt]

you may wish to ensure compatibility and evaluate functional overlap with DISDRODB

[scollis]

Yes, we have been looking at this. The main thing is DISDRODB's main "thing" is the DB. This is extra to what we want. We'll chat with the Devs and see how we can work together.

[josephhardinee]

Sorry for a delayed response, I wanted to think on this for a little bit.
When I originally left ARM, I found my name stripped off multiple DOIs for work we'd completed (as well as others that had left ARM) so I have had bad experiences with DOE-ARM on maintaining credit. Attempts to get PyDSD supported internally were not met with success and we had to get it included in other activities, or get usually get outside funding to develop it. the one group that's historically been supportive, at least as a consumer of it, from Brookhaven hasn't been part of this conversation either.

That history might read differently if ARM had a track record of contributing back to the project and were a natural successor to take it over. I too would like to see it used and acknowledge both @nguy and myself have been busy and not particularly active but you've said it is used actively internally but I had to scroll back almost a decade to find a PR from ARM. Despite that active use, ARM hasn't historically contributed improvements back.

As it stands, the ask here is essentially: hand over ownership, or ARM will build something separate with no offer of collaboration or support in either direction. I may be misreading the intent, and text often lands harsher than it's meant to, but that's how it came across to me.

So to directly answer the question: no, I don't think a leadership handover makes sense right now. I would genuinely welcome ARM getting involved as a contributor, and if that relationship develops over time, governance is absolutely something we could revisit. If ARM feels that didn't give it enough skin in the game or thought it might delay their PRs, we could discuss getting an ARM representative commit rights.

As a project we are always accepting of PRs and collaborators. This is of course also open source so you absolutely have the right to fork and proceed (internally or publicly) as long as the license terms of original contributors are respected; We chose an open and permissive license specifically to allow that.

[scollis]

Thanks @josephhardinee I appreciate your position.

If PyDSD is, indeed, accepting PRs then just leaving it here may be an option. I will consult with @AdamTheisen

[AdamTheisen]

Hey @josephhardinee and @scollis , completely agree on the DOI part. The current method of how ARM handles DOIs and authors is not ideal. Because it's at the instrument code level and not datastream either past mentors get replaced on DOIs by current mentors or past mentors continue to get credit on DOIs for efforts past their time in ARM. It is never the intent to strip credit where credit is due. Happy to discuss and look back through DOI to make sure you get the appropriate credit if you have specifics to point me to.

As for PyDSD, it's definitely not the intent to have this come across as an ultimatum. First and foremost, it would be great if we could contribute and modernize PyDSD while keeping the history and credit to those that have created and developed it so far. That was the first path forward on our list of options. The big question from our side was how involved you can be with open-source with your current work and whether or not you would be able to respond to PRs. The last option we wanted to consider was rolling something new in the event we wouldn't be able to effectively contribute here and get it merged in.

I would very much be open to the idea of building up the collaboration here through PRs and see where that takes us with the goal of having someone from ARM as part of the governance structure. Of course we would have to have discussions in ARM and go through the design review so things could change but I do hate duplication of code and effort so this seems like a good path forward.

Open to thoughts and comments from others here though.