diff --git a/doc/DRAGONS/changes.rst b/doc/DRAGONS/changes.rst index 591085f79..1a9f0aef9 100644 --- a/doc/DRAGONS/changes.rst +++ b/doc/DRAGONS/changes.rst @@ -16,10 +16,12 @@ New Features ------------ Full support for the reduction of GHOST data. - This is based on the external GHOSTDR package, with improvements and - changes to the names and scope of primitives to better align with the + This is based on the external GHOSTDR package, with important improvements. + Includes changes to the names and scope of primitives to better align with the other instrument recipes. +Support for new GMOS-S CCDs installed in late 2023. + Improvements ------------ **astrodata.wcs** @@ -28,7 +30,16 @@ Improvements * Support for reading and writing tabular wavelength information to/from FITS. -**geminidr** +**astrodata.provenance** + +* Renamed the ``PROVHISTORY`` table to ``HISTORY``, and changed wording in the +code from "provenance history" to simply "history". + +**astrodata.fits** + +* Support reading ASCII tables when opening FITS files with astrodata + +**geminidr.core** * Creation of new ``skip_primitive`` parameter, e.g., ``reduce -p skyCorrectFromSlit:skip_primitive=True`` which allows any @@ -40,51 +51,47 @@ Improvements ``reduce -p ADUToElectrons:write_outputs=True`` which will write to disk the outputs of the primitive. +* Allow input files to ``shiftImages`` to recognize tabs or multiple + whitespaces as the delimiter + +**geminidr.gsaoi*** + +* Modification to the `nostack` science recipe to not "store" the image but + rather continue and detect sources in the images in anticipation of the likely + stacking that will follow later. The output images will have the + `_sourcesDetected` suffix rather than the `_image` suffix. + +**recipe_system.cal_service** + +* Whitespace now allowed in directory paths (if quoted), e.g., + ``databases = "~/.my dragons/dragons.db"`` + Bug fixes --------- -**geminidr** +**geminidr.core** * Set default ``calculateSensitivity.bandpass`` parameter to 0.001 nm to better handle pure spectra in flux density units. -* Allow ``display`` to handle non-standard extention names, which did not +* Allow ``display`` to handle non-standard extension names, which did not work as intended. +**geminidr.gmos** + +* Fix the QE model selection for the GMOS-S EEV CDDs. **recipe_system** * Set the ``engineering`` flag to False for all data stored in the local calibration database, to ensure that it can be retrieved. +Compatibility +------------- +**geminidr.interactive** -3.1.1 -===== - -Improvements ------------- - -**astrodata.provenance** - -* Renamed the ``PROVHISTORY`` table to ``HISTORY``, and changed wording in the -code from "provenance history" to simply "history". - -**geminidr.core** - -* Allow input files to ``shiftImages`` to recognize tabs or multiple - whitespaces as the delimiter - -**recipe_system.cal_service** - -* Whitespace now allowed in directory paths (if quoted), e.g., - ``databases = "~/.my dragons/dragons.db"`` - -Bug Fixes ---------- - -**geminidr.gmos** +* The interactive tools are now compatible with and require bokeh v3 and above. -* Fix the QE model selection for the GMOS-S EEV CDDs. 3.1.0 diff --git a/doc/DRAGONS/conf.py b/doc/DRAGONS/conf.py index ee2bef143..acec80664 100644 --- a/doc/DRAGONS/conf.py +++ b/doc/DRAGONS/conf.py @@ -294,6 +294,7 @@ def setup(app): .. |GNIRSImgTut| replace:: :raw-html:`GNIRS Keyhole Imaging Data Reduction Tutorial` .. |GMOSLSTut| replace:: :raw-html:`GMOS Longslit Data Reduction Tutorial` +.. |GHOSTTut| replace:: :raw-html:`GHOST Data Reduction Tutorial` .. |ADMaster| replace:: :raw-html:`Astrodata Manual` .. |ADCheat| replace:: :raw-html:`AstroData Cheat Sheet` diff --git a/doc/DRAGONS/dragonsteam.rst b/doc/DRAGONS/dragonsteam.rst index afb2c97f7..4a53d5d28 100644 --- a/doc/DRAGONS/dragonsteam.rst +++ b/doc/DRAGONS/dragonsteam.rst @@ -10,13 +10,14 @@ Current DRAGONS Team ==================== (Alphabetical order) +* Dan Avner * Daniel Berke * Paul Hirst -* David Jones * Kathleen Labrie * Chris Simpson * Olesja Smirnova * Monika Soraisam +* D.J. Teal * James Turner * Bill Vacca diff --git a/doc/DRAGONS/index.rst b/doc/DRAGONS/index.rst index 15452a499..c73a012a2 100644 --- a/doc/DRAGONS/index.rst +++ b/doc/DRAGONS/index.rst @@ -34,8 +34,9 @@ Tutorials - Reducing data with DRAGONS * |GSAOIImgTut| * |GNIRSImgTut| -**Longslit Spectroscopy** +**Spectroscopy** +* |GHOSTTut| * |GMOSLSTut| DRAGONS Manuals diff --git a/doc/DRAGONS/releasenotes.rst b/doc/DRAGONS/releasenotes.rst index de8dfe380..20d4bb6cf 100644 --- a/doc/DRAGONS/releasenotes.rst +++ b/doc/DRAGONS/releasenotes.rst @@ -6,6 +6,37 @@ Release Notes ************* +V3.2.0 +====== + +This release adds: + +* Fully integrated support for GHOST +* Support for the new GMOS-S CCDs that were installed in late 2023 + +The release also includes a number of other improvements and bug fixes. +See the :ref:`changes` for all the details. + +With this release, DRAGONS offers support for: + +Science Quality reduction + * GMOS imager + * NIRI imager + * GSAOI imager + * F2 imager + * GMOS longslit spectrograph (including nod-and-shuffle) + * GHOST spectrograph + +For imaging and GMOS longslit spectroscopy, this software should be used +instead of the Gemini IRAF package. + +To install DRAGONS, please follow the installation guide provided in the +Recipe System User Manual: + + |RSUserInstall|. + + + V3.1.0 ====== diff --git a/recipe_system/doc/rs_UsersManual/install.rst b/recipe_system/doc/rs_UsersManual/install.rst index 4a56996ad..adf1f4d01 100644 --- a/recipe_system/doc/rs_UsersManual/install.rst +++ b/recipe_system/doc/rs_UsersManual/install.rst @@ -4,6 +4,18 @@ https://www.anaconda.com/distribution/#download-section +.. |miniforge_link| raw:: html + + https://github.com/conda-forge/miniforge + +.. |miniforgelinux| raw:: html + + Miniforge3-Linux-x86_64.sh + +.. |miniforgemacosx| raw:: html + + Miniforge3-MacOSX-x86_64.sh + .. |geminiiraf_link| raw:: html https://www.gemini.edu/observing/phase-iii/reducing-data/gemini-iraf-data-reduction-software @@ -14,77 +26,83 @@ Installation ************ -The Recipe System is distributed as part of DRAGONS. DRAGONS is available +The Recipe System is the automation part of DRAGONS. DRAGONS is available as a conda package. The installation instructions below will install all the necessary dependencies. The use of the ``bash`` shell is recommended. -Install Anaconda -================ +Install Miniforge +================= -Download and install Anaconda ------------------------------ -If you already have Anaconda installed, you can skip this step and go to -the :ref:`Install DRAGONS ` section below. If not, then your -first step is to get and install Anaconda. You can download it from the -Anaconda website. We recommend clicking on the "Get Additional Installers" -instead of using the green Download button as it will allow you to do a finer -selection. Here we show how to use the "Command Line Installer" +We recommend using Miniforge instead of Anaconda. It is smaller and install +contents from conda-forge which is now our main channel for python packages. +The overall size of the installation will therefore be much smaller than +a full Anaconda installation. The instructions below are for Miniforge. -+-----------------------------------------+ -| **Download Anaconda**: |anaconda_link| | -+-----------------------------------------+ +Anaconda will work just fine too if you prefer or have a business need for +it. We just don't provide instructions for its installation. Miniforge or +Anaconda will not impact the DRAGONS installation. -.. warning:: M1 MacOS Users!!! DRAGONS is not yet built with the M1 - architecture. The x86 build will work anyway. But you have to be careful - when you install Anaconda. +Download and install Miniforge +------------------------------ +If you already have Miniforge installed (or Anaconda and don't want to move +to Miniforge), you can skip this step and go to the +:ref:`Install DRAGONS ` section below. - We recommend that you use the "64-bit Command Line Installer", ie. the - non-M1 version. This version is the x86 (aka Intel) build. When using - that version, by default, x86 binaries of the various conda packages will - be installed. +If not, then your first step is to get and install Miniforge. You can download +it from the Miniforge github page. The direct links to the Linux and Mac +installer are provided in the table below. - M1 can run x86 binaries via the Rosetta interface. It is seamless. ++--------------------------------------------------+ +| **Miniforge Page**: |miniforge_link| | ++--------------------------------------------------+ +| **Linux x64_86 Installer**: |miniforgelinux| | ++--------------------------------------------------+ +| **MacOSX x64_86 Installer**: |miniforgemacosx| | ++--------------------------------------------------+ -The current version of DRAGONS has been tested with Python 3.9 and Python 3.10. -We recommend that you install the standard Python 3 version of Anaconda, the -specific Python version can be adjusted later, if necessary. -If you have downloaded the graphical installer, follow the graphical installer -instructions. Install in your home directory. It should be the default. +.. warning:: arm64 MacOS Users!!! (That's M1/M2/M etc) DRAGONS is not yet + built with the arm64 architecture. Some dependencies are also either not + available for arm64 or not reliable. The x86 build will work. All you + need to do is install the x64_86 version of Miniforge (or Anaconda). + That way, the x64_86 binaries will automatically be seeked, and found. -If you have downloaded the command-line installer (recommended), type the -following in a terminal, replacing the ``.sh`` file name to the name of the -file you have downloaded. The ``/bin/bash -l`` line is not needed if you are -already using bash. The command-line installer allows for more customization -of the installation. + arm64 machines can run x86 binaries via the Rosetta interface. It is + seamless. + +The current version of DRAGONS has been tested with Python 3.10. At the time +of this writing, Miniforge installs Python 3.10 by default. (Other version +of Python an subsequently be installed.) + +To install, run the installer that you have downloaded +Type the following in a terminal, replacing the ``.sh`` file name to the name +of the file you have downloaded. The ``/bin/bash -l`` line is not needed if +you are already using bash. :: - $ /bin/bash -l - $ chmod a+x Anaconda3-2022.10-MacOSX-x86_64.sh - $ ./Anaconda3-2022.10-MacOSX-x86_64.sh + $ /bin/bash Miniforge3-MacOSX-x86_64.sh (``$`` indicates the terminal prompt.) -.. note:: To prevent the Anaconda "base" environment from loading - automatically, do:: +.. note:: To prevent the "base" environment from loading automatically, do:: $ conda config --set auto_activate_base false -Verify Anaconda installation ----------------------------- +Verify Miniforge installation +----------------------------- We recommend the use of the ``bash`` shell:: $ /bin/bash -l -Make sure that ``~/anaconda3/bin/conda`` is in your ``PATH`` by doing:: +Make sure that ``~/miniforge3/bin/conda`` is in your ``PATH`` by doing:: $ which conda -The Anaconda installer should have added conda configurations to the +The Miniforge installer should have added conda configurations to the ``~/.bash_profile`` for you. If ``conda`` is not found, try:: $ source ~/.bash_profile @@ -93,11 +111,11 @@ If ``activate`` is still not found, you might have to add ``export PATH=~/anaconda3/bin:$PATH`` to your ``~/.bash_profile`` using your favorite text editor, and run the ``source`` command above again. -.. note:: Sometimes the Anaconda installer will install the software in +.. .. note:: Sometimes the Anaconda installer will install the software in ``~/anaconda3`` instead of simply ``~/anaconda``. Just check in your home directory which one of the tow possibilities was used. -The code Anaconda adds to the .bash_profile will automatically activate +The code Miniforge adds to the .bash_profile will automatically activate anaconda. To activate or deactivate Anaconda manually:: $ conda activate @@ -108,7 +126,7 @@ anaconda. To activate or deactivate Anaconda manually:: Install DRAGONS =============== -With Anaconda installed and ready to go, now we can install DRAGONS and +With Miniforge installed and ready to go, now we can install DRAGONS and the necessary dependencies. Add conda-forge and the Gemini channel. Those channels host the conda packages @@ -118,11 +136,17 @@ that we will need. $ conda config --add channels conda-forge $ conda config --add channels http://astroconda.gemini.edu/public - $ conda config --set channel_priority disabled + +The content of the `~/.condarc` file should look like this:: + + channels: + - http://astroconda.gemini.edu/public + - conda-forge + - defaults The next step is to create a virtual environment and install the DRAGONS software and its dependencies in it. The name of the environment can be -anything you like. Here we use "dragons" as the name and we install +anything you like. Here we use "dragons" as the name and we request Python 3.10. :: @@ -137,17 +161,17 @@ You will need to activate the environment whenever you start a new shell. If you are planning to use it all the time, you might want to add the command to your ``.bash_profile``, after the "conda init" block. -.. note:: +.. .. note:: For Linux users only. - As a side note, if you are going to use PyRAF regularly, for example to +.. As a side note, if you are going to use PyRAF regularly, for example to reduce Gemini data not yet supported in DRAGONS, you should install the ``iraf-all`` and ``pyraf-all`` conda packages as well. - $ conda create -n geminiconda python=3.10 iraf-all pyraf-all ds9 dragons +.. .. $ conda create -n geminiconda python=3.10 iraf-all pyraf-all ds9 dragons - DRAGONS and the Recipe System do not need IRAF or PyRAF, however. See the - Gemini website for information on how to configure IRAF (|geminiiraf_link|) +.. DRAGONS and the Recipe System do not need IRAF or PyRAF, however. See the +.. Gemini website for information on how to configure IRAF (|geminiiraf_link|) .. _configure: @@ -186,11 +210,11 @@ configurations:: Add this line to the .bash_profile: export IMTOOLRC=~/.imtoolrc -While not specific at all to DRAGONS, it is recommended to increase the -Operating System limit on the number of opened files. We have seen an increase -in reports of the error "Too many open files." +It is recommended to increase the Operating System limit on the number of +opened files. We have seen an increase in reports of the error +"Too many open files" when reducing spectroscopy data. -In your .bash_profile, add the following line to overcome that OS limitation:: +In your `.bash_profile`, add the following line to overcome that OS limitation:: ulimit -n 1024 @@ -217,7 +241,7 @@ are compiled and loaded:: This will print the reduce help to the screen. -If you have Gemini FITS files available, you can test that the Recipe System +If you have Gemini FITS files available, you can test that DRAGONS is functioning as expected as follow (replace the file name with the name of your file):: @@ -254,3 +278,5 @@ If all is well, you will see something like:: Wrote N20180106S0700_prepared.fits in output directory reduce completed successfully. + +