Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
fd75f36
Added function for creating dask map of selection function using SSI …
tmcornish Apr 21, 2026
e4c7ac2
Began writing stage for creating map of selection function
tmcornish Apr 21, 2026
7f8dfa9
Uncertainty on selection function now also output by make_dask_select…
tmcornish Apr 23, 2026
f153026
Basic selection function and uncertainties computed and stored
tmcornish Apr 23, 2026
fac42a0
Pixels without SSI now display hp.UNSEEN, not NaN
tmcornish Apr 27, 2026
413a53f
Pixels without SSI now also display hp.UNSEEN in the uncertainty map
tmcornish Apr 28, 2026
2fbf40d
Prelimary working version of stage which models selection function ac…
tmcornish Apr 28, 2026
5feaeb1
Added potential TODO
tmcornish Apr 28, 2026
2163c4a
Changed variable names
tmcornish Apr 29, 2026
efe9791
Detection and injection count maps are now saved in outputs
tmcornish Apr 29, 2026
32fe3e0
TXModelSelectionFunction is now dask-compatible
tmcornish Apr 29, 2026
e2f7788
Removed print statements
tmcornish Apr 29, 2026
e3a0c92
Model parameters and their covariances now saved to file
tmcornish Apr 29, 2026
b090bbe
Fixed bug in which attempts to apply any non-zero mask threshold woul…
tmcornish Apr 29, 2026
519fbbf
Added option to apply mask threshold after degrading to desired resol…
tmcornish Apr 29, 2026
9901eff
Removed custom ud_grade function since upgrading resolution not neces…
tmcornish Apr 29, 2026
a75b00a
Moved where coarse mask cut is applied and added a cut in number of i…
tmcornish Apr 29, 2026
18fe420
Removed TODO
tmcornish Apr 29, 2026
d1cd7b5
Edited comments
tmcornish Apr 30, 2026
3e36da8
Removed uncertainty calculation from TXSelectionFunctionSSIMaps
tmcornish Apr 30, 2026
bc55df7
Uncertainties on measured selection function now calculated in situ b…
tmcornish Apr 30, 2026
c2e5f84
Added total injection counts as function og mag threshold to outputs …
tmcornish Apr 30, 2026
4eaae97
TXModelSelectionFunction now models a selection function for each det…
tmcornish Apr 30, 2026
6b75ad5
Deleted TXSelectionFunctionSSIMaps since now using outputs from TXAux…
tmcornish Apr 30, 2026
09af75e
Removed unused function
tmcornish Apr 30, 2026
c85708d
Changed ordering of groups/datasets in model_info output file
tmcornish Apr 30, 2026
90728a0
Tidying up
tmcornish Apr 30, 2026
e5a084d
Removed unused function
tmcornish Apr 30, 2026
0c36714
Removed import of deleted function
tmcornish Apr 30, 2026
2097b26
Removed deleted function from mapping/__init__.py
tmcornish Apr 30, 2026
11af4f7
Weighted mean now used when degrading selection function
tmcornish May 7, 2026
27bb13b
Now uses hsp.operations.product_intersection when determining valid p…
tmcornish May 7, 2026
149ba55
Removed now unused mask_pix
tmcornish May 7, 2026
da16179
Added possible argument values to sel_func_err_type description
tmcornish May 7, 2026
fde4dd7
Added more detailed docstrings for all methods
tmcornish May 7, 2026
8294343
Changed built-in all() to dask.array.all()
tmcornish May 8, 2026
5075994
Added option to specify for which bins the selection function will be…
tmcornish May 8, 2026
c6449d6
Added error handling for case where there are zero valid pixels in th…
tmcornish May 8, 2026
de74cca
Removed print statement
tmcornish May 8, 2026
c73ddb8
err_type now included as an argument in most methods (avoids need to …
tmcornish May 11, 2026
703cef6
Circumvents costly matrix multiplication when computing uncertainties…
tmcornish May 11, 2026
4e440f6
Changed name of class to TXPredictDensitySelectionFunction
tmcornish May 11, 2026
b2093d1
Added example pipeline and config
tmcornish May 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions examples/ssi/config_ssi_sel_func.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Values in this section are accessible to all the different stages.
# They can be overridden by individual stages though.
global:
# This is read by many stages that read complete
# catalog data, and tells them how many rows to read
# at once
chunk_rows: 100_000
# These mapping options are also read by a range of stages
pixelization: healpix
nside: 256
sparse: true # Generate sparse maps - faster if using small areas


TXIngestSSIDetectionDESBalrog:
name: TXIngestSSIDetectionDESBalrog

TXIngestSSIMatchedDESBalrog:
name: TXIngestSSIMatchedDESBalrog

TXAuxiliarySSIMaps:
name: TXAuxiliarySSIMaps
mag_delta: 0.5
smooth_det_frac: false
min_depth: 23
max_depth: 24
smooth_window: 2.0


TXPredictDensitySelectionFunction:
name: TXPredictDensitySelectionFunction
systmaps_dir: /global/cfs/cdirs/lsst/groups/WLSS/users/tcornish/balrog_txpipe_example/survey_properties/
degree: 2
mask_thresh: 0.
mask_thresh_coarse: 0.
inj_count_thresh: 1
sel_func_err_type: gaussian
block_size: 2_000
bins_to_model: [0,1]

43 changes: 43 additions & 0 deletions examples/ssi/pipeline_ssi_sel_func.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Stages to run
stages:
- name: TXIngestSSIDetectionDESBalrog
- name: TXIngestSSIMatchedDESBalrog
- name: TXAuxiliarySSIMaps
- name: TXPredictDensitySelectionFunction

modules: txpipe

# Where to put outputs
output_dir: /pscratch/sd/t/tcornish/txpipe_out/

# How to run the pipeline: mini, parsl, or cwl
launcher:
name: mini
interval: 1.0

# Where to run the pipeline: cori-interactive, cori-batch, or local
site:
name: local
max_threads: 2

# configuration settings
config: examples/ssi/config_ssi_sel_func.yml

inputs:
# See README for paths to download these files

#NERSC locations of public DES balrog data
balrog_detection_catalog: /global/cfs/cdirs/lsst/groups/WLSS/users/tcornish/balrog_txpipe_example/balrog_detection_catalog_sof_y3-merged_v1.2.fits
balrog_matched_catalog: /global/cfs/cdirs/lsst/groups/WLSS/users/tcornish/balrog_txpipe_example/balrog_matched_catalog_sof_y3-merged_v1.2.fits
mask: /global/cfs/cdirs/lsst/groups/WLSS/users/tcornish/balrog_txpipe_example/des_y3_mask.hdf5
#aux_ssi_maps: /home/cornisht/cloned_repos/TXPipe/data/example/outputs_ssi_des_depth/aux_ssi_maps.hdf5

fiducial_cosmology: data/fiducial_cosmology.yml

# if supported by the launcher, restart the pipeline where it left off
# if interrupted
resume: false
# where to put output logs for individual stages
log_dir: data/example/outputs_ssi_des_sel_func/logs/
# where to put an overall parsl pipeline log
pipeline_log: data/example/outputs_ssi_des_sel_func/log.txt
Loading
Loading