JP-3755: Remove unused options and add unit tests to SOSS extraction algorithm #9000
Conversation
…tests for that function
| # In both orders, the errors on the replaced pixels are roughly | ||
| # half of the errors on the original good pixels | ||
| # There are enough replaced pixels here (~30) that small-number statistics cannot account for this | ||
| # The reason is because the code chooses the lower error between the two nearest-flux | ||
| # data points, and since the errors in our tests are uncorrelated with the flux values, | ||
| # this leads to a factor-of-2 decrease | ||
| # It's not clear to me that picking the smaller of two error values is the right thing to do | ||
| # but that behavior is documented |
There was a problem hiding this comment.
Wondering if this is actually the intended behavior or not, because it seems like a questionable choice to me (although admittedly I don't know very much). Any opinions @hover2pi ?
|
@hover2pi I'm attaching the first of a few comparisons I plan to make between the output spectra from this PR branch vs the current release branch (jwst==1.16.1, which corresponds to DMS build 11.1.1). The test data here were some SOSS data I had lying around from fixing #8166. The attached 6 spectra show three integrations (indices 71, 72, and 73, where 72 contains the data causing the problem from the aforementioned issue). So the filename The upshot here is that the typical differences between my PR branch and the current main branch are typically of order parts per ten thousand. The residuals are a bit larger at very long wavelengths (beyond 2.5 um) and it's not initially obvious to me why, but the differences are still what I would consider very small. Let me know your thoughts, and whether you think I should investigate anything in these residuals further or just deem them "close enough". I plan to try this with the SUBSTRIP96 data you mentioned on #8983 next. Random side note, the regularization factors (both initial guess and best-fitting solution) are larger in the PR branch by around half an order of magnitude and sometime 1 order of magnitude. I don't think the results should be sensitive to the specific choice of regularization factor; the code tests in 1/4 order-of-mag increments, and the likelihoods seem very similar between adjacent factors. This was what I was concerned about initially RE numerical differences, but as shown above it doesn't seem to matter much for the outputs. |
|
Same plots for some substrip96 data, these are the first two integrations from jw03596001001_03102_00001-seg001. They should be interpreted the same way; first plot shows integration 0 order 1, second plot shows integration 0 order 2, third shows int 1 order 1, 4th shows int 1 order 2. For order 1, same story as above: the differences are very small, except for the flux drop that is still present for jwst==1.16.1 but was fixed by your previous PR. The differences for order 2 are much, much more substantial. It's not clear which of these looks more like the expected spectrum. I did make one change to the "Model the remaining part of order 2" portion of the code that may have caused this, which set some 0-valued pixels to NaN. I'll look into this further of course, but perhaps that was the wrong thing to do. |
There was a problem hiding this comment.
@emolter Everything looks good to me! I tried to respond to all your comments that asked questions. I do need to do a little reading on how the ATOCA algorithm works from Antoine's paper and then may have better answers. Ultimately, if the results are within our tolerances after removing a whole bunch of stuff, I think you were successful. We'll be revisiting several aspects of the code in the coming year, particularly by replacing the specprofile and speckernel reference files with pastasoss functionality. As it stands, I think it's in great shape. Thanks!
|
@emolter Regarding the differences in the extracted order 2 spectra from the SUBSTRIP96 data between the two branches, I'll definitely take a closer look. It would be great to understand why it changed but ultimately order 2 is not really used from that data. Between the heavy contamination from order 1 at long wavelengths and the order 2 throughput dropping to almost 0 around 1.2um, it's not too worrisome. |
|
regression tests stared here: https://github.com/spacetelescope/RegressionTests/actions/runs/12712456299 |
| @@ -277,7 +276,7 @@ def find_spectral_order_index(refmodel, order): | |||
| return -1 | |||
|
|
|||
|
|
|||
| def get_soss_traces(refmodel, pwcpos, order, subarray, interp=True): | |||
| def _get_soss_traces(refmodel, pwcpos, order, subarray): | |||
There was a problem hiding this comment.
@hover2pi Is there any expectation that external users may wish to use pastasoss but not ATOCA? If so, would they use this function, or is the NIRISS team still planning to direct users to the pastasoss standalone package?
If there is a chance users may wish to call this function independently, we may want to reconsider the name change.
|
Pending some small discussions and clean regression tests, this looks good to me! 👏 |
Resolves JP-3755
Closes #8804
This PR addresses reducing the technical debt of the SOSS ATOCA algorithm. There are a lot of changed lines, so I'm annotating this PR with comments during a self-review. The high-level changes are:
There are several outstanding items to complete before this can be merged:
Tasks
Build 11.3(use the latest build if not sure)no-changelog-entry-needed)changes/:echo "changed something" > changes/<PR#>.<changetype>.rst(see below for change types)docs/pageokify_regteststo update the truth filesnews fragment change types...
changes/<PR#>.general.rst: infrastructure or miscellaneous changechanges/<PR#>.docs.rstchanges/<PR#>.stpipe.rstchanges/<PR#>.datamodels.rstchanges/<PR#>.scripts.rstchanges/<PR#>.fits_generator.rstchanges/<PR#>.set_telescope_pointing.rstchanges/<PR#>.pipeline.rststage 1
changes/<PR#>.group_scale.rstchanges/<PR#>.dq_init.rstchanges/<PR#>.emicorr.rstchanges/<PR#>.saturation.rstchanges/<PR#>.ipc.rstchanges/<PR#>.firstframe.rstchanges/<PR#>.lastframe.rstchanges/<PR#>.reset.rstchanges/<PR#>.superbias.rstchanges/<PR#>.refpix.rstchanges/<PR#>.linearity.rstchanges/<PR#>.rscd.rstchanges/<PR#>.persistence.rstchanges/<PR#>.dark_current.rstchanges/<PR#>.charge_migration.rstchanges/<PR#>.jump.rstchanges/<PR#>.clean_flicker_noise.rstchanges/<PR#>.ramp_fitting.rstchanges/<PR#>.gain_scale.rststage 2
changes/<PR#>.assign_wcs.rstchanges/<PR#>.badpix_selfcal.rstchanges/<PR#>.msaflagopen.rstchanges/<PR#>.nsclean.rstchanges/<PR#>.imprint.rstchanges/<PR#>.background.rstchanges/<PR#>.extract_2d.rstchanges/<PR#>.master_background.rstchanges/<PR#>.wavecorr.rstchanges/<PR#>.srctype.rstchanges/<PR#>.straylight.rstchanges/<PR#>.wfss_contam.rstchanges/<PR#>.flatfield.rstchanges/<PR#>.fringe.rstchanges/<PR#>.pathloss.rstchanges/<PR#>.barshadow.rstchanges/<PR#>.photom.rstchanges/<PR#>.pixel_replace.rstchanges/<PR#>.resample_spec.rstchanges/<PR#>.residual_fringe.rstchanges/<PR#>.cube_build.rstchanges/<PR#>.extract_1d.rstchanges/<PR#>.resample.rststage 3
changes/<PR#>.assign_mtwcs.rstchanges/<PR#>.mrs_imatch.rstchanges/<PR#>.tweakreg.rstchanges/<PR#>.skymatch.rstchanges/<PR#>.exp_to_source.rstchanges/<PR#>.outlier_detection.rstchanges/<PR#>.tso_photometry.rstchanges/<PR#>.stack_refs.rstchanges/<PR#>.align_refs.rstchanges/<PR#>.klip.rstchanges/<PR#>.spectral_leak.rstchanges/<PR#>.source_catalog.rstchanges/<PR#>.combine_1d.rstchanges/<PR#>.ami.rstother
changes/<PR#>.wfs_combine.rstchanges/<PR#>.white_light.rstchanges/<PR#>.cube_skymatch.rstchanges/<PR#>.engdb_tools.rstchanges/<PR#>.guider_cds.rst