Investigate sparse layer autograd

[ilan-gold]

I just had a thought about something:

https://github.com/laminlabs/modlyn/blob/main/modlyn/models/_simple_logreg_model.py#L47

Uses a normal torch linear layer that assumes dense input. It might be worth invesitgating for peformance reasons using one that accepts sparse based potentially on https://github.com/rusty1s/pytorch_sparse but there might be other / more comprehensive solutions out there. Maybe there is none, feel free to close!

Just figured I'd document the thought!

[falexwolf]

Thanks for the comment! Things are not at all optimized right now; I hope we get to this.