Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP updates to AP and Solar System #38

Merged
merged 8 commits into from
Aug 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion access.tex
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,4 @@ \subsection{Community Brokers }
Alerts are fully world-public and will be accessible via one or more of the nine Rubin-endorsed Community Brokers\footnote{See \url{https://www.lsst.org/scientists/alert-brokers}}.
During the commissioning period, Rubin will work with the Community Brokers to integrate them \citedsp{rtn-010}.
Community access to early alerts will depend on the readiness of the Community Brokers.
At this stage, we are expecting the first alerts from commissioning to become available via Community Brokers sometime following System First Light \citedsp{rtn-061}.
At this stage, we are expecting the first alerts from commissioning to become available via Community Brokers near the start of the LSST survey (\S \ref{sec:alerts_start}).
33 changes: 14 additions & 19 deletions alertproduction.tex
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ \subsection{Processing Overview}
Substantial validation will be required to determine when to build incremental templates to maximize the net throughput of Early Science.
Nevertheless our goal is to enable Alert Generation to begin over at least a subset of the survey area as soon as the data are scientifically useful.

Coadding multiple images enables artifact rejection \citeds{DMTN-080} and is formally required due to the noise-level requirements placed on the DM system.
Coadding multiple images enables artifact rejection \citedsp{DMTN-080} and is formally required due to the noise-level requirements placed on the DM system.
Additionally, the LSST survey is heavily dithered, so without coadding many images onto a common sky plane it is both difficult and inefficient to obtain image differences for a new pointing from past single images.
Finally, single-image templates do not permit removal of artifacts, transients, and moving objects from the template, creating additional false positive sources in the resulting differences.

Expand All @@ -55,29 +55,24 @@ \subsection{Supporting Incremental Template Generation}
\end{itemize}


\subsection{Alert Generation during Commissioning}
\subsection{Beginning Alert Production} \label{sec:alerts_start}

Due to the need to verify the instrument characteristics, template quality, and image differencing and Real/Bogus performance, real-time alerts will not be immediately available during the commissioning period.
However, the incremental template generation and alert production processes will be tested and optimized during commissioning with LSSTCam, resulting in a set of prompt data products, including alerts.
Broker teams will be given access to these alerts for development purposes as soon as they are produced to understand their characteristics and to help to validate their quality, rather than to enable rapid followup and Early Science.
The Alert and Prompt Product databases will be made available for query prior to the DP2 data release.
A variety of technical, scientific, and organizational criteria must be met prior to the release of the first public alerts from LSSTCam.
The prerequisites, which include formal agency approval, are summarized in \citeds{RTN-061}.
Accordingly, alert production is expected to begin around the time of the Science Validation Surveys with LSSTCam and the start of the full LSST survey.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps add a statement about not being full-scale routine AP until Y2?

Incremental template generation, difference imaging, and Real/Bogus pipelines will be tested and optimized during commissioning with ComCam and LSSTCam prior to first alert release.
These efforts will contribute to data release DIA products released in DP1 and DP2.
Full-volume, full-fidelity alert production is expected in the second year of the LSST survey after DR1.

During commissioning templates will be generated incrementally over the maximal sky area supported by the available observations.
By the end of the commissioning period, coadd templates for use in difference imaging will only be available for $\approx$ 10\% of the sky.
Generating templates over a wide area is not an explicit goal of commissioning; however, where possible, if commissioning observations are agnostic to pointing and filter, we would endeavour to choose a pointing and filter that maximizes building templates to enable early science.

Rubin aims to scale up alert production during commissioning with the aim of beginning routine Alert Production as soon as is feasible following System First light.
Once begun, Alert Production will then proceed continuously into the full LSST survey.
Alerts generated during commissioning may be produced with higher latency.
The commissioning period also provides an excellent opportunity to investigate how many visits in a given band are sufficient to construct a usable template.

Table~\ref{tab:prompt-data-products} lists the various alert and prompt processing data products currently planned at each phase of alert production during commissioning and the first two years of the LSST survey.
Phase 1 covers the commissioning system optimization and science validation periods and phase 2 covers early survey operations.
Rubin aims to begin routine Alert Production as soon as is feasible and scale up alert production as more templates become available.
Once begun, Alert Production will proceed continuously through the full LSST survey.
Alerts generated prior to DR1 may be produced with higher latency and/or lower fidelity.
All Rubin alerts sent to community alert brokers are world-public.

\input{tables/rubin_early_pp_scenario}
%\begin{table}[ht]
%\centering
%\includegraphics[width=0.9\linewidth]{figures/Prompt-products}
%\caption{Summary of prompt data products expected during commissioning and year 1 survey observations.}
%\label{tab:prompt-data-products}
%\end{table}
Other data products produced during Alert Production (processed visit and difference images, DIA catalogs in the Prompt Products Database, direct image catalogs) may be accessed by Data Rights holders through the RSP.
Access to these products may be delayed early in the survey (Figure \ref{fig:commissioning-es-schedule}).
3 changes: 2 additions & 1 deletion datapreview.tex
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,9 @@ \subsection{Data Preview 1}
possible following the end of either ComCam on sky or System First Light (\S~\ref{ssec:commissioning-milestones}), to enable the community to prepare to work with LSST data.
In the current schedule, which re-implements on-sky data-taking with ComCam for 2 months, DP1 would be based on science-grade data taken from the 2 months of ComCam data.

In both DP1 scenarios, the exact processing pipelines and data products are still under discussion.
In both DP1 scenarios, the exact processing pipelines and data products are still to be determined and depend strongly on the dataset that emerges from the commissioning on-sky observing program.
At minimum, DP1 will deliver visit-level calibrated images and catalogs to enable initial studies of observational and instrumental effects.
leannep marked this conversation as resolved.
Show resolved Hide resolved
Rubin will do its best to deliver the maximum of data products possible with the commissioning on-sky dataset, however at this time, for DP1, all other data products described in the DPDD remain a stretch goal.
Table~\ref{tab:data-preview-summary} presents a current best idea of the data products that might comprise DP1.

A final decision on the dataset on which DP1 will be based will be taken in February 2024 following the Project's confirming decision about ComCam on-sky data-taking (\S~\ref{ssec:commissioning-schedule})
Expand Down
33 changes: 18 additions & 15 deletions tables/rubin_early_dr_scenario.tex
Original file line number Diff line number Diff line change
Expand Up @@ -3,44 +3,47 @@
\fontsize{6}{10}\selectfont
\setlength{\tabcolsep}{6pt} % Default value: 6pt
{\renewcommand{\arraystretch}{1.2}
\begin{tabular}{|l|c|c|c|c|c|c|c|c|}
\begin{tabular}{|l|c|c|c|c|c|c|c|c|c|}
\hline
\multicolumn{9}{|l|}{{\fontsize{9}{12}\selectfont \color{RubinDarkTeal}\textbf{Rubin Early Science -- Data Release Scenario}}} \\\hline\hline
\multicolumn{10}{|l|}{{\fontsize{9}{12}\selectfont \color{RubinDarkTeal}\textbf{Rubin Early Science -- Data Release Scenario}}} \\\hline\hline
\multirow{2}{*} {} &
\tiny Jun 2021 & \tiny Jun 2022 & \tiny Jun 2023 &
\tiny \makecell{Oct 2024 -- \\ Jul 2025} &
\tiny \makecell{start Jun--\\Nov 2025} &
\tiny \makecell{ Nov 2025 -- \\ May 2026} &
\tiny \makecell{ May 2026 -- \\ Jan 2027} &
\tiny \makecell{ May 2027 -- \\ Jan 2028} &
\tiny \makecell{ May 2028 -- \\ Nov 2028}
\\ \cline{2-9}
\\ \cline{2-10}

& \textbf{DP0.1} & \textbf{DP0.2} & \textbf{DP0.3} & \textbf{DP1} & \textbf{DP2} & \textbf{DR1} & \textbf{DR2} & \textbf{DR3}
\\\cline{2-9}
& \textbf{DP0.1} & \textbf{DP0.2} & \textbf{DP0.3} & \textbf{DP1} & \textbf{AP} & \textbf{DP2} & \textbf{DR1} & \textbf{DR2} & \textbf{DR3}
\\\cline{2-10}
\multirow{3}{*}{\parbox{0.1\linewidth}{\vspace{0.2cm} \textbf{Data Product}}} &
\rotatebox[origin=c]{90}{\tiny \makecell{\textbf{DC2} \\ \textbf{Simulated} \\\ \textbf{Sky Survey} }} &
\rotatebox[origin=c]{90}{\tiny\makecell{\textbf{Reprocessed} \\ \textbf{DC2 Survey} }} &
\rotatebox[origin=c]{90}{\tiny\makecell{\textbf{Solar System} \\ \textbf{PPDB} \\ \textbf{Simulation}}} &
\rotatebox[origin=c]{90}{\tiny\makecell{\textbf{ComCam} \\ \textbf{ or early} \\ \textbf{ LSSTCam} \\ \textbf{Data}}} &
\rotatebox[origin=c]{90}{\tiny\makecell{\textbf{ComCam} \\ \textbf{Data}}} &
\rotatebox[origin=c]{90}{\tiny\makecell{\textbf{Realtime} \\ \textbf{LSSTCam}\\ \textbf{Data}}} &
\rotatebox[origin=c]{90}{\tiny\makecell{\textbf{LSSTCam} \\ \textbf{ Science} \\ \textbf{Validation} \\ \textbf{Data}}} &
\rotatebox[origin=c]{90}{\tiny\makecell{\textbf{LSST First } \\ \textbf{6 Months} \\ \textbf{Data}}} &
\rotatebox[origin=c]{90}{\tiny\makecell{\textbf{LSST } \\ \textbf{Year 1} \\ \textbf{ Data}}} &
\rotatebox[origin=c]{90}{\tiny\makecell{\textbf{LSST} \\ \textbf{Year 2} \\ \textbf{ Data}}}
\\\cline{2-9 } \hline
\\\cline{2-10} \hline

Raw Images & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \arrayrulecolor{gray}\hline
DRP Processed Visit Images and Visit Catalogs & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\\hline
DRP Coadded Images & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\\hline
DRP Object and ForcedSource Catalogs & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
DRP Difference Images and DIASources & -- & \mycirc[RubinDarkTeal] & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
DRP ForcedSource Catalogs including DIA output & -- & \mycirc[RubinDarkTeal] & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
Raw Images & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \arrayrulecolor{gray}\hline
Processed Visit Images and Catalogs & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\\hline
Coadded Images & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & -- & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\\hline
Object and ForcedSource Catalogs & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & -- & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
Difference Images and DIASources & -- & \mycirc[RubinDarkTeal] & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
ForcedSource Catalogs including DIA & -- & \mycirc[RubinDarkTeal] & -- & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
%PP Processed Visit Images & -- & -- & -- & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
%PP Difference Images & -- & -- & -- & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
%PP Catalogs & -- & -- & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
% PP SSP Catalogs & -- & -- & \mycirc[RubinDarkTeal] & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\ \hline
DRP SSP Catalogs & -- & -- & -- & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\
Alerts & -- & -- & -- & -- & \mycirc[RubinDarkTeal] & -- & -- & -- & -- \\ \hline
SSP Catalogs & -- & -- & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] & \mycirc[RubinDarkTeal] \\
\arrayrulecolor{black}\hline
\end{tabular}}
\caption{Summary of Data Release data products expected in each data preview and early LSST data release.}
\caption{Summary of data products expected in each early LSST data release.}
\label{tab:data-preview-summary}
\end{table}
leannep marked this conversation as resolved.
Show resolved Hide resolved
38 changes: 0 additions & 38 deletions tables/rubin_early_pp_scenario.tex

This file was deleted.

5 changes: 3 additions & 2 deletions tables/rubin_ops_timeline.tex
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,9 @@
& & &&&&&&&&&& % 2028
\\\hline

\tiny Start of Regular Alert Production & \tiny Jun 2025 -- Nov 2025
% &&&&&& % 2024

\tiny Start of Alert Production & \tiny Jun 2025 -- Nov 2025
% &&&&&& % 2024
& &&&& & \cellcolor{TLOrange3} & \cellcolor{TLOrange3} & \cellcolor{TLOrange2} & \cellcolor{TLOrange2} &\cellcolor{TLOrange2}&\cellcolor{TLOrange3}& % 2025 \cellcolor{TLOrange3} & % 2025
& & &&&&&&&&&& % 2026
& & &&&&&&&&&& % 2027
Expand Down
Loading