intro.tex

\section{Introduction}

Within the Vera C.\ Rubin Observatory \citep{2019ApJ...873..111I} the Data Management (DM) team was tasked to stand up operable, maintainable, quality services to deliver high-quality LSST data products for science and education, all on time and within reasonable cost.
DM is responsible for provided the tools necessary to take the bits generated by the telescope and turn them in to science ready products.

 See also the Rubin Observatory  Data Management System \citep{2017ASPC..512..279J,2022arXiv221113611O}


\subsection{Science Drivers}
The astronomical size and complexity of the expected Rubin data drives many of the architectural choices made for the DM system. The following table highlights some of the key numbers that have influenced choices in DM.

\begin{deluxetable}{llc}
\tablecaption{Rubin Key Numbers driving DM architectural choices\label{tab:dm_keynumbers}}

\tablehead{\colhead{Parameters} & \colhead{Number} & \colhead{Unit}}

\startdata
N Objects & 40 billion & -- \\
N Alerts per image & 10 000 & -- \\
N Alerts per night & 10 million & -- \\
N Images per night & 1000 & -- \\
\enddata

\end{deluxetable}

\subsection{Technical Challenges}
The operational goal of Rubin Observatories Legacy Survey of Space and time is to produce an optical/near-IR survey of half the sky in ugrizy bands to r 27.5 (36 nJy) based on 825 visits over a 10-year period. It is a deep wide fast survey.
Each Rubin image is around 8GB and we take more than one per minute or about 1000 per night.
Add the $\approx$ 450 calibration exposures each day and it means about 20TB of data has to be shipped from Chile to SLAC on a daily basis.
Alerts are to be produced in under 2 minutes with a  goal of doing them in 1 minute which gives us a
challenging transmission and prompt processing time window (see \autoref{sec:dataacquisition}).

Over the 10 year survey we estimate having 2.75M on sky images and at least 1M more calibrations totaling about 50PB.
As we grow this must be reprocessed each year to produce the data releases (see \autoref{sec:dataproduction}.