Proper tracking of flagged times #328
Conversation
There was a problem hiding this comment.
Thank you for hunting this down!! I'm not worried about the PS changes, they look small and reasonable and this is clearly the more correct thing to do. I have one comment about the polarization aspect that doesn't need to block this but I want to understand a little better.
| ; If any polarization is fully flagged, then count that time as flagged | ||
| time_use[ti] *= total((*vis_weight_ptr[pol_i])[*, bin_offset[ti]:bin_offset[ti+1]-1] > 0) GT 0 |
There was a problem hiding this comment.
This might need to be revisited in the case that Dara developed for HERA where antenna flagging can be per pol. I don't think we want to support different per-time flagging per pol if both pols are included, but we do want to support entirely flagging one pol of an antenna and not flagging the other pol. We can push that to another PR if we want just wanted to note the potential conflict.
|
@miguelfmorales you should see this. |
This fix addresses issue #321, where the auto calibration was using times which should have been flagged.
This underlying root cause is that the time_use array was not checked against the weights array. In the case of SSINS, the weights for an entire time step will flagged, but this was not propagating to the time_use array in FHD. The time_use array is used a few circumstances, the two major instances being 1) auto calibration averaging and 2) noise calculation. Given that this affects the results in a few places, there is an overall change in the PS.
Below is a test against a control for an observation that had 4 SSINS flagged times. Differences are small and somewhat hard to interpret. Not sure if we want to pursue this differences deeper or if we are happy to take the fact that this is the proper time_use handling.