From 121cfecbbb2b1e1a973a38365679ac4ff10e173e Mon Sep 17 00:00:00 2001 From: Jaladh Singhal Date: Thu, 17 Oct 2024 15:40:57 -0700 Subject: [PATCH 1/3] Remove layout setup code, and show coverage Clean up query string representation --- tutorials/firefly/NEOWISE_light_curve_demo.md | 71 ++++++++----------- 1 file changed, 31 insertions(+), 40 deletions(-) diff --git a/tutorials/firefly/NEOWISE_light_curve_demo.md b/tutorials/firefly/NEOWISE_light_curve_demo.md index f41a79e..884058f 100644 --- a/tutorials/firefly/NEOWISE_light_curve_demo.md +++ b/tutorials/firefly/NEOWISE_light_curve_demo.md @@ -39,8 +39,8 @@ For documentation on the firefly client visit https://caltech-ipac.github.io/fir ## Imports - *firefly_client FireflyClient* - Python API to Firefly for displaying tables, images and charts -- *firefly_client.plot* for visualizing the light curve in the client - *astropy.utils.data* for downloading the catalog data via TAP query +- *urllib.parse* for converting regular query string to url-safe string ```{code-cell} ipython3 # Uncomment the next line to install dependencies if needed. @@ -49,13 +49,13 @@ For documentation on the firefly client visit https://caltech-ipac.github.io/fir ```{code-cell} ipython3 from firefly_client import FireflyClient -import firefly_client.plot as ffplt import astropy.utils.data +import urllib.parse ``` ## Step 1 -Instantiate the client via and view it in a different tab in the web browser. +Instantiate the client and view it in a different tab in the web browser. In this example, we use the IRSA Viewer - a public firefly server. The firefly server can also be run locally, e.g. via a Firefly Docker image obtained from https://hub.docker.com/r/ipac/firefly/tags/. @@ -64,10 +64,9 @@ url = 'https://irsa.ipac.caltech.edu/irsaviewer' # url='http://127.0.0.1:8080/firefly' # if you have firefly server running locally (preferably through docker) fc = FireflyClient.make_client(url) -ffplt.use_client(fc) ``` -You can re-initizialize the viewer to return to a clean slate with [`reinit_viewer`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.reinit_viewer). +You can re-initizialize the viewer to return to a clean state with [`reinit_viewer`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.reinit_viewer). ```{code-cell} ipython3 # fc.reinit_viewer(); # The semi-colon suppresses the output of the method when ran @@ -75,68 +74,60 @@ You can re-initizialize the viewer to return to a clean slate with [`reinit_view ## Step 2 -Setup the layout of viewer and TAP search the 'Known Solar System Object Possible Association List' catalog from the NEOWISE-R database. The specific target we are looking for is minor planet `558 Carmen`. We can query this target using a TAP search through IRSA; the `table_url` is broken down as follows: +TAP search the 'Known Solar System Object Possible Association List' catalog from the NEOWISE-R database. The specific target we are looking for is minor planet `558 Carmen`. We can query this target using a TAP search through IRSA; the `table_url` is broken down as follows: -- We want to search the data through IRSA, which supports TAP querying, and we want it streamed directly to us via a synchronous search:
"https://irsa.ipac.caltech.edu/TAP/sync?"

-- Next, we want to structure query to only retrieve (558) Carmen data from the NEOWISE-R 'Known Solar System Object Possible Association List' catalog. The table name of the catalog can be found using [IRSAViewer](https://irsa.ipac.caltech.edu/irsaviewer/?__action=layout.showDropDown&view=MultiTableSearchCmd) and clicking the **VO TAP Search** tab and changing the 'Project' to **neowiser**. We query all columns of data and we search the target by its object id, which is its name, and use the 'like' condition to only write (558) with a wildcard:
"QUERY=SELECT+*+FROM+neowiser_p1ba_mch+AS+n+WHERE+n.objid+like+'(558)%'" +- We want to search the data through IRSA, which supports TAP querying, and we want it streamed directly to us via a synchronous search:
"https://irsa.ipac.caltech.edu/TAP/sync"

+- Next, we want to structure query to only retrieve (558) Carmen data from the NEOWISE-R 'Known Solar System Object Possible Association List' catalog. The table name of the catalog can be found using [IRSAViewer](https://irsa.ipac.caltech.edu/irsaviewer/?__action=layout.showDropDown&view=MultiTableSearchCmd) and clicking the **VO TAP Search** tab and changing the 'Project' to **neowiser**. We query all columns of data and we search the target by its object id, which is its name, and use the 'like' condition to only write (558) with a wildcard as shown in the cell below. Construction of the query can be found in the [`IRSA TAP documentation page`](https://irsa.ipac.caltech.edu/docs/program_interface/TAP.html). -We first add a cell to the layout that will hold the table; this cell is shown at row = 0, col = 0, with width = 4, height = 2. Once the cell is created, we can request the necessary data from the catalog and display the data as a table using the [`show_table`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_table) method. +```{code-cell} ipython3 +BASE_URL = "https://irsa.ipac.caltech.edu/TAP/sync" +QUERY = """ +SELECT * +FROM neowiser_p1ba_mch AS n +WHERE n.objid LIKE '(558)%' +""" +table_url = f"{BASE_URL}?QUERY={urllib.parse.quote_plus(QUERY)}" +table_url +``` + +Now, we can request the necessary data from the catalog and display the data as a table in the Firefly client, using the [`show_table`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_table) method. Alternatively, we can download data from the catalog using [`astropy.utils.data.download_file`](https://docs.astropy.org/en/stable/api/astropy.utils.data.download_file.html) and upload it to the Firefly client shown in the cell below the first method. ```{code-cell} ipython3 -r = fc.add_cell(0, 0, 4, 2, 'tables', 'main') - -if r['success']: - table_url = ("https://irsa.ipac.caltech.edu/TAP/sync?QUERY=SELECT+*+FROM+neowiser_p1ba_mch+AS+n+WHERE+n.objid+like+'(558)%'") - fc.show_table(table_url, tbl_id='tableneo', title='558 Carmen NeoWise Catalog', page_size=50) +fc.show_table(table_url, tbl_id='tableneo', title='558 Carmen NeoWise Catalog', page_size=50) ``` ```{code-cell} ipython3 -# r = fc.add_cell(0, 0, 4, 2, 'tables', 'main') +# tablename = astropy.utils.data.download_file(table_url, timeout=120, cache=True) +# file = fc.upload_file(tablename) +# fc.show_table(file, tbl_id='tableneo', title='558 Carmen Catalog', page_size=50) +``` -# if r['success']: -# table_url = ("https://irsa.ipac.caltech.edu/TAP/sync?QUERY=SELECT+*+FROM+neowiser_p1ba_mch+AS+n+WHERE+n.objid+like+'(558)%'") -# tablename = astropy.utils.data.download_file(table_url, timeout=120, cache=True) +Note that along with the table, firefly also displays the coverage and chart associated with the table. It overlays colored squares for each row of the table onto a HiPS image, because the table contains recognizable celestial coordinates. It also creates a scatter plot of ra and dec from the table. -# file = fc.upload_file(tablename) -# fc.show_table(file, tbl_id='tableneo', title='558 Carmen Catalog', page_size=50) -``` ++++ ## Step 3 After retrieving the data and displaying it in the client, we can now create a light curve by plotting the Modified Julian Date ('mjd') in the abscissa and the magnitude from band W1 ('w1mpro') in the ordinate. We also flip the ordinate to accurately display magnitude. ```{code-cell} ipython3 -r = fc.add_cell(2, 0, 2, 2, 'plot-image', 'light-curve') -if r['success']: - status = fc.show_xyplot(tbl_id='tableneo', xCol='mjd', yCol='w1mpro', yOptions='flip') +fc.show_xyplot(tbl_id='tableneo', xCol='mjd', yCol='w1mpro', yOptions='flip') ``` ## Step 4 -Finally, we can overlay the catalog of data in the table onto a HiPS image using the [`show_coverage`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_coverage) method. - -You will notice that there are colored squares that signify where the object was observed based on the RA and Dec given in the catalog. - -```{code-cell} ipython3 -r = fc.add_cell(2, 2, 2, 2, 'catalog-image', 'target') -if r['success']: - fc.show_coverage(); -``` - -Alternatively, we can queue a HiPS image using the method [`show_hips`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_hips). However, this method requires target coordinates for the object you want to analyze. +Finally, we can overlay the catalog of data in the table onto a HiPS image of our choice, using the method [`show_hips`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_hips). However, this method requires target coordinates for the object you want to analyze. ```{code-cell} ipython3 target='229.851396;-9.720647;EQ_J2000' viewer_id = 'hipsDiv' hips_url = 'http://alasky.u-strasbg.fr/AllWISE/RGB-W4-W2-W1' -r = fc.add_cell(2, 2, 2, 2, 'catalog-image', 'target') -if r['success']: - status = fc.show_hips(viewer_id=viewer_id, plot_id='aHipsID1-1', hips_root_url = hips_url, +fc.show_hips(viewer_id=viewer_id, plot_id='aHipsID1-1', hips_root_url = hips_url, Title='HiPS-WISE', WorldPt=target) ``` @@ -146,7 +137,7 @@ Firefly allows you to visualize data for specific targets. In conjuction with As 1. We import all necessary modules to create a Firefly client and to download the catalog of data for our target. -2. We start the client in our web browser and prep the layout to appropiately display our tables, plots and images. +2. We start the client in our web browser to appropiately display our tables, plots and images. 3. We use the TAP schema to display the data for our target — [`558 Carmen`](https://irsa.ipac.caltech.edu/irsaviewer/?__action=table.search&request=%7B%22startIdx%22%3A0%2C%22SearchMethod%22%3A%22AllSky%22%2C%22RequestedDataSet%22%3A%22NEOWISE%20Reactivation%20Database%22%2C%22id%22%3A%22GatorQuery%22%2C%22tbl_id%22%3A%22tbl_id-cf48-45%22%2C%22META_INFO%22%3A%7B%22title%22%3A%22WISE-neowiser_p1ba_mch%20(AllSky)%22%2C%22tbl_id%22%3A%22tbl_id-cf48-45%22%2C%22tbl_pref_key%22%3A%22WISE-neowiser_p1ba_mch%22%7D%2C%22catalogProject%22%3A%22WISE%22%2C%22catalog%22%3A%22neowiser_p1ba_mch%22%2C%22constraints%22%3A%22objid%20like%20%27%25(558)%20Carmen%25%27%22%2C%22pageSize%22%3A100%7D&options=%7B%22backgroundable%22%3Atrue%2C%22pageSize%22%3A100%7D) — via a table and visualize such data through charts. @@ -161,5 +152,5 @@ Firefly allows you to visualize data for specific targets. In conjuction with As +++ **Author:** Eric Bratton II (IRSA Scientist) in conjunction with the IRSA Science Team
-**Updated On:** 2024-07-31
+**Updated On:** 2024-10-17
**Contact:** irsasupport@ipac.caltech.edu or https://irsa.ipac.caltech.edu/docs/help_desk.html From 16767f517f434d51e4f7db97e860835555d4445d Mon Sep 17 00:00:00 2001 From: Jaladh Singhal Date: Thu, 17 Oct 2024 15:42:06 -0700 Subject: [PATCH 2/3] Add WCS locking, and image titles --- tutorials/firefly/SEDs_in_Firefly.md | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/tutorials/firefly/SEDs_in_Firefly.md b/tutorials/firefly/SEDs_in_Firefly.md index d2a1e17..6246718 100644 --- a/tutorials/firefly/SEDs_in_Firefly.md +++ b/tutorials/firefly/SEDs_in_Firefly.md @@ -553,7 +553,7 @@ phot_tbl This notebook assumes that [jupyter_firefly_extensions](https://github.com/Caltech-IPAC/jupyter_firefly_extensions) has been installed, and that the Firefly server to use has been specified before Jupyterlab was started. -If everything has been properly configured, executing the next cell will bring up a Firefly tab in Jupyterlab with the message "Firefly Ready": +If everything has been properly configured, executing the next cell will bring up a Firefly tab in Jupyterlab with the welcome message. ```{code-cell} ipython3 # Uncomment to use the jupyter_firefly_extensions @@ -566,7 +566,6 @@ fc = FireflyClient.make_client(url="https://irsa.ipac.caltech.edu/irsaviewer") In the event that there are problems with the tab opened above, run the below command to obtain a web link that can be opened in a browser directly: ```{code-cell} ipython3 -# Temporary work-around to Firefly error for Slate and/or Internet connection problems fc.display_url() ``` @@ -577,7 +576,7 @@ tblpath = "./phot_tbl.fits" phot_tbl.write(tblpath, format="fits", overwrite=True) ``` -Upload the table to Firefly: +Upload the table (FITS file) to Firefly: ```{code-cell} ipython3 tval = fc.upload_file(tblpath) @@ -667,9 +666,9 @@ j_fname, h_fname, k_fname = get_2mass_images(target) ``` ```{code-cell} ipython3 -fc.show_fits(fc.upload_file(j_fname), plot_id="2MASS J") -fc.show_fits(fc.upload_file(h_fname), plot_id="2MASS H") -fc.show_fits(fc.upload_file(k_fname), plot_id="2MASS K") +fc.show_fits(fc.upload_file(j_fname), plot_id="2MASS J", title="2MASS J") +fc.show_fits(fc.upload_file(h_fname), plot_id="2MASS H", title="2MASS H") +fc.show_fits(fc.upload_file(k_fname), plot_id="2MASS K", title="2MASS K") ``` ```{code-cell} ipython3 @@ -684,14 +683,14 @@ fc.set_pan(plot_id="2MASS K", x=target.ra.deg, y=target.dec.deg, coord="j2000") ``` -In the future, the user should be able to implement turning on "Align and Lock by WCS" using the Python API as part of this notebook. +For turning on "Align and Lock by WCS" in these images, there's no dedicated method in firefly but you can use this workaround (using low-level API): -+++ - -The current alternative is to use the following link to launch IRSA Finder Chart in a separate browser and turn on that function there: -https://irsa.ipac.caltech.edu/applications/finderchart/servlet/api?mode=getResult&locstr=270.599026+-24.147018+Equ+J2000&subsize=0.0167&sources=2MASS&DoSearch=true +```{code-cell} ipython3 +fc.dispatch('ImagePlotCntlr.wcsMatch', + payload=dict(matchType='Standard', lockMatch=True)) +``` -In that Firefly display, it should be possible to turn on "Align and Lock by WCS" in the image toolbar. +Note that zooming or panning one image, will do the same in others. +++ @@ -705,7 +704,7 @@ In that Firefly display, it should be possible to turn on "Align and Lock by WCS **Author:** David Shupe (IRSA Scientist) and Joyce Kim (IRSA Scientist) in conjunction with the IRSA Science Team -**Updated On:** 2024-07-31 +**Updated On:** 2024-10-17 **Contact:** irsasupport@ipac.caltech.edu or https://irsa.ipac.caltech.edu/docs/help_desk.html From 8cfe2b3566d6266e5c6e8de5d78c617dc2af4429 Mon Sep 17 00:00:00 2001 From: Jaladh Singhal Date: Fri, 18 Oct 2024 16:16:38 -0700 Subject: [PATCH 3/3] Grammar fixes in SED notebook --- tutorials/firefly/SEDs_in_Firefly.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tutorials/firefly/SEDs_in_Firefly.md b/tutorials/firefly/SEDs_in_Firefly.md index 6246718..8e3e4d4 100644 --- a/tutorials/firefly/SEDs_in_Firefly.md +++ b/tutorials/firefly/SEDs_in_Firefly.md @@ -683,14 +683,14 @@ fc.set_pan(plot_id="2MASS K", x=target.ra.deg, y=target.dec.deg, coord="j2000") ``` -For turning on "Align and Lock by WCS" in these images, there's no dedicated method in firefly but you can use this workaround (using low-level API): +For turning on "Align and Lock by WCS" in these images, there's no dedicated method in Firefly but you can use this workaround (using low-level API): ```{code-cell} ipython3 fc.dispatch('ImagePlotCntlr.wcsMatch', payload=dict(matchType='Standard', lockMatch=True)) ``` -Note that zooming or panning one image, will do the same in others. +Note that zooming or panning one image will do the same in the other images as well. +++