Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different NPIXELS counts when only resolution matrices are different #322

Open
segasai opened this issue Jan 14, 2025 · 1 comment
Open

Comments

@segasai
Copy link

segasai commented Jan 14, 2025

Hi,

I have a bit of a puzzle. I have two coadd files that are identical (i.e. FLUX,MASK,IVAR's are the same) with the exception of the resolution matrices. When the files are run through redrock, NPIXELS in the redrock files are different by 2000 or so. (the script is given below). I don't quite see how this could happen. I could see that the NPIXELS comes from ivar>0 count in redrock, but I don't see anywhere in the code where the resolution matrix is used to mask the ivar. Also the resolution matrix is sensible everywhere I believe.

Do you have any theories of what could be going on ?
I tried to look here

if comm is not None:

But, it's not easy to figure out what's going on there with all the arrays on different mpi ranks.

Thanks

Script (to be run on nersc)

import astropy.io.fits as pyfits
import numpy as np
import os

f1 = '/global/cfs/cdirs/desi/spectro/redux/koposov/fluxcalib_withfix/tiles/cumulative/80658/20210220/coadd-3-80658-thru20210220.fits'
f2 = '/global/cfs/cdirs/desi/spectro/redux/koposov/fluxcalib_nofix/tiles/cumulative/80658/20210220/coadd-3-80658-thru20210220.fits'
for a in 'BRZ':
    for k in ['FLUX', 'IVAR', 'MASK']:
        A1, A2 = [pyfits.getdata(_, a + '_' + k) for _ in [f1, f2]]
        # checking equality 
        print(np.all(A1 == A2))
outputs = ['tmp1.fits', 'tmp2.fits']
cmds = [
    f'rrdesi -i {_}  --targetids 39633220106454453 --outfile {__}'
    for _, __ in zip([f1, f2], outputs)
]

[os.system(_) for _ in cmds]
print('npix1', pyfits.getdata(outputs[0])['NPIXELS'])
print('npix2', pyfits.getdata(outputs[1])['NPIXELS'])

Script output:

True
True
....
Total run time: 15.2 seconds
npix1 [6026]
npix2 [7928]
@segasai segasai changed the title Different NPIXELS counts while only resolution matrices are different Different NPIXELS counts when only resolution matrices are different Jan 14, 2025
@segasai
Copy link
Author

segasai commented Jan 14, 2025

Further look into the code revealed that the cause is
this line where the sum of the resolution matrix columns (or rows, depending on how you look at it) is required to be bigger than 0.99

w = np.asarray(R.sum(axis=1))[:,0]<constants.min_resolution_integral

I am not sure I understand the rationale for this. I would understand if the limit was like 0.1 or something like that, but not 0.99.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant