Fix large-cube memory issues with channel-wise processing

[EOakes]

Addresses Issue #276 by adding large-cube-safe, channel-wise processing in a few steps that were previously loading too much data into memory.

Changes:

• casaMaskingRoutines.py: add channel-by-channel path in noise_for_cube().
• casaMosaicRoutines.py: add channel-by-channel paths in generate_weight_file() and mosaic_aligned_data().
• scConvolution.py: enable huge operations in coverage_collapser(); add channel-by-channel path in smooth_cube().
• handlerDerived.py: add huge cube flag for channel-wise processing.

Why:
These changes prevent memory-related crashes on very large ALMA cubes while preserving existing workflow behavior.

Validation:
Tested on NGC 4945 CO cube in CASA 6.6.1.17 where these steps previously failed due to memory limits.

[thomaswilliamsastro]

I've made some quick edits here, since there were some things that looked duplicated to me and a missing import. You should double-check this all runs on your end. There's one comment I have that might need a bit more work

[thomaswilliamsastro]

Also gonna page @e-koch on this

[e-koch]

Looks good @EOakes; thank you! Added a couple small suggestions

[codecov]

Codecov Report

❌ Patch coverage is 0.55402% with 359 lines in your changes missing coverage. Please review.
✅ Project coverage is 6.27%. Comparing base (a242249) to head (b6bb546).

Files with missing lines	Patch %	Lines
phangsPipeline/casaMosaicRoutines.py	0.40%	244 Missing ⚠️
phangsPipeline/casaMaskingRoutines.py	1.33%	74 Missing ⚠️
phangsPipeline/scConvolution.py	0.00%	38 Missing ⚠️
phangsPipeline/handlerDerived.py	0.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##           master    #323      +/-   ##
=========================================
- Coverage    6.39%   6.27%   -0.13%     
=========================================
  Files          38      38              
  Lines       14882   15204     +322     
  Branches     3560    3647      +87     
=========================================
+ Hits          952     954       +2     
- Misses      13916   14236     +320     
  Partials       14      14              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[thomaswilliamsastro]

I mentioned this to @EOakes on Slack, but I actually ran into a case where this is necessary last week. Can confirm these fixes work on my end, though it would be nice to not force the plane-by-plane if we don't need it, since in my case I/O is quite slow and this slows things down a bit

[thomaswilliamsastro]

Also, check_getchunk_putchunk_memory_issue relies on CASA, which we don't insist is installed for the derived pipeline. I'd suggest either reading in the fits file we have here and getting the data shape, or just getting the size of the file with os.path.getsize and if it's bigger than a few GB, flip to huge cube. Don't know if @e-koch has any better thoughts here

[EOakes]

Makes sense. Any threshold preference? check_getchunk_putchunk_memory_issue uses 2880x2880x393 (~3e9) but I'm not sure where that's from. We could also check the available memory with something like psutil but that may be over-engineered.

[thomaswilliamsastro]

I would maybe stick with those same numbers just for consistency. Loading in a fits file that big should be fine, I'd guess it'd be order 10GB or so

[EOakes]

Sounds good. Just pushed a commit using os.path.getsize - chose this over reading in fits file since it's a little simpler and doesn't have additional dependencies. But can switch to fits implementation if preferred.

[thomaswilliamsastro]

Seems fine to me, I'm not sure exactly the mapping between image size and no. of bytes but hey, this is about 3GB so seems like the regime where we should be flipping the switch. I'm happy with this, any other worries @e-koch ?

[e-koch]

I think @1054 worked out the threshold where the memory issues were happening, so best to stick with that threshold throughout. Otherwise LGTM!