Skip to content

Make gaus_2d faster (2)#274

Open
AlexKurek wants to merge 3 commits into
lofar-astron:masterfrom
AlexKurek:Make-gaus_2d-faster-(2)
Open

Make gaus_2d faster (2)#274
AlexKurek wants to merge 3 commits into
lofar-astron:masterfrom
AlexKurek:Make-gaus_2d-faster-(2)

Conversation

@AlexKurek
Copy link
Copy Markdown
Contributor

before (but after #273)

    10935    0.200    0.000    0.204    0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)
    10935    0.205    0.000    0.208    0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)
    10935    0.222    0.000    0.226    0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)
    10935    0.196    0.000    0.200    0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)

after:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    10322    0.173    0.000    0.177    0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)
    10322    0.167    0.000    0.171    0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)
    10322    0.167    0.000    0.171    0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)
    10322    0.167    0.000    0.172    0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)
    10322    0.165    0.000    0.169    0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)

~18,4% faster. A lot is done in place now. I think this is now as fast as it gets.
There are very rare differences in the .gaul file at 13th decimal place, but this is normal.

@AlexKurek
Copy link
Copy Markdown
Contributor Author

After 291f1e0 it gets faster by another ~24% :
10322 0.128 0.000 0.153 0.000 /home/akurek/miniconda3/envs/pybdsf-313_local_temp/lib/python3.13/site-packages/bdsf/functions.py:202(gaus_2d)

The .gaul file is OK.

@gmloose
Copy link
Copy Markdown
Collaborator

gmloose commented May 6, 2026

Can you provide me the input file(s) you're using to do the profiling? I'd like to experiment a bit myself, and would like to be able to make a fair comparison. Also, did you just call gaus_2d in isolation, or is this profile part of a different function call? Anyway, please provide all the necessary information to reproduce your experiments.

@AlexKurek
Copy link
Copy Markdown
Contributor Author

AlexKurek commented May 6, 2026

Since it is an optimisation, I assumed that it would be best to use the latest Python and Numpy, as these are constantly being updated to improve performance. I was not able to build against Python 3.14 nor using pip install . -e. This is the best I can do for now:

Paste all this to the terminal in one go. It will remove conda env and cloned repo from previous experiment and prepare new conda env for installation anew.

cd ~
rm -rf PyBDSF
conda deactivate

conda remove --name pybdsf-313_local_temp --all -y
conda create -n pybdsf-313_local_temp python=3.13 -y
conda activate pybdsf-313_local_temp
conda install -c conda-forge cmake ninja boost libboost-python-devel numpy scipy matplotlib astropy gfortran_linux-64 gcc_linux-64 gxx_linux-64 -y
# Tools for communication Python <-> CMake <-> Fortran
pip install scikit-build setuptools wheel meson setuptools_scm
export CMAKE_ARGS="-DBoost_PYTHON_LIBRARY_RELEASE=$CONDA_PREFIX/lib/libboost_python313.so \
                  -DBoost_PYTHON_INCLUDE_DIR=$CONDA_PREFIX/include \
                  -DBoost_NUMPY_LIBRARY_RELEASE=$CONDA_PREFIX/lib/libboost_numpy313.so \
                  -DBOOST_ROOT=$CONDA_PREFIX"
export FFLAGS="-std=legacy"
export FCFLAGS="-std=legacy"
git clone https://github.com/lofar-astron/PyBDSF.git
cd PyBDSF

Now paste your experimental changes using a text editor to the clonned repo. Then paste all this in one go to the terminal. It will install PyBDSF, move to a different folder to execute a profilled run and display the result for map_2d.

pip install .
cd ..
cd bench_pybdsf
python -m cProfile -o out.prof run_pybdsf.py
echo "stats map_2d" | python -m pstats out.prof

run_pybdsf.py is attached. Here is a link for FITS file (127 MB). Please tell me when I can remove it from the server.
https://www.oa.uj.edu.pl/A.Kurek/NGC6048_9-MFS-image.fits

Please note that the total running time is not stable:

PyBDSF running time: 0:2:38
PyBDSF running time: 0:2:30
PyBDSF running time: 0:2:42
PyBDSF running time: 0:2:35
PyBDSF running time: 0:2:33

As far as I know this is normal for high level languages, as there is a garbage collector going all over the code etc. Thats why I was always doing the profiling few times.

At the first time you can also display full profilling results:

python -m pstats out.prof
sort cumulative
stats 60

to see what should be optimised.

One comment about 326df50. At first I was thinking that it would be optmial to let it compute on positive values as long as possible. But I have profilled your suggestion and it turned out that there is no difference.

run_pybdsf.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants