Skip to content

Commit

Permalink
Merge pull request #36 from jaladh-singhal/update-firefly-tut
Browse files Browse the repository at this point in the history 
Update firefly tutorials as per the latest firefly-client API
  • Loading branch information
@bsipocz
bsipocz authored Oct 28, 2024
2 parents fa18b12 + 8cfe2b3 commit f3e1bd9
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 53 deletions.
71 changes: 31 additions & 40 deletions tutorials/firefly/NEOWISE_light_curve_demo.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ For documentation on the firefly client visit https://caltech-ipac.github.io/fir
## Imports

- *firefly_client FireflyClient* - Python API to Firefly for displaying tables, images and charts
- *firefly_client.plot* for visualizing the light curve in the client
- *astropy.utils.data* for downloading the catalog data via TAP query
- *urllib.parse* for converting regular query string to url-safe string

```{code-cell} ipython3
# Uncomment the next line to install dependencies if needed.
Expand All @@ -49,13 +49,13 @@ For documentation on the firefly client visit https://caltech-ipac.github.io/fir

```{code-cell} ipython3
from firefly_client import FireflyClient
import firefly_client.plot as ffplt
import astropy.utils.data
import urllib.parse
```

## Step 1

Instantiate the client via and view it in a different tab in the web browser.
Instantiate the client and view it in a different tab in the web browser.

In this example, we use the IRSA Viewer - a public firefly server. The firefly server can also be run locally, e.g. via a Firefly Docker image obtained from https://hub.docker.com/r/ipac/firefly/tags/.

Expand All @@ -64,79 +64,70 @@ url = 'https://irsa.ipac.caltech.edu/irsaviewer'
# url='http://127.0.0.1:8080/firefly' # if you have firefly server running locally (preferably through docker)
fc = FireflyClient.make_client(url)
ffplt.use_client(fc)
```

You can re-initizialize the viewer to return to a clean slate with [`reinit_viewer`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.reinit_viewer).
You can re-initizialize the viewer to return to a clean state with [`reinit_viewer`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.reinit_viewer).

```{code-cell} ipython3
# fc.reinit_viewer(); # The semi-colon suppresses the output of the method when ran
```

## Step 2

Setup the layout of viewer and TAP search the 'Known Solar System Object Possible Association List' catalog from the NEOWISE-R database. The specific target we are looking for is minor planet `558 Carmen`. We can query this target using a TAP search through IRSA; the `table_url` is broken down as follows:
TAP search the 'Known Solar System Object Possible Association List' catalog from the NEOWISE-R database. The specific target we are looking for is minor planet `558 Carmen`. We can query this target using a TAP search through IRSA; the `table_url` is broken down as follows:

- We want to search the data through IRSA, which supports TAP querying, and we want it streamed directly to us via a synchronous search: <br>"https://<!---->irsa.ipac.caltech.edu/TAP/sync?"<br><br>
- Next, we want to structure query to only retrieve (558) Carmen data from the NEOWISE-R 'Known Solar System Object Possible Association List' catalog. The table name of the catalog can be found using [IRSAViewer](https://irsa.ipac.caltech.edu/irsaviewer/?__action=layout.showDropDown&view=MultiTableSearchCmd) and clicking the **VO TAP Search** tab and changing the 'Project' to **neowiser**. We query all columns of data and we search the target by its object id, which is its name, and use the 'like' condition to only write (558) with a wildcard: <br>"QUERY=SELECT+*+FROM+neowiser_p1ba_mch+AS+n+WHERE+n.objid+like+'(558)%'"
- We want to search the data through IRSA, which supports TAP querying, and we want it streamed directly to us via a synchronous search: <br>"https://<!---->irsa.ipac.caltech.edu/TAP/sync"<br><br>
- Next, we want to structure query to only retrieve (558) Carmen data from the NEOWISE-R 'Known Solar System Object Possible Association List' catalog. The table name of the catalog can be found using [IRSAViewer](https://irsa.ipac.caltech.edu/irsaviewer/?__action=layout.showDropDown&view=MultiTableSearchCmd) and clicking the **VO TAP Search** tab and changing the 'Project' to **neowiser**. We query all columns of data and we search the target by its object id, which is its name, and use the 'like' condition to only write (558) with a wildcard as shown in the cell below.

Construction of the query can be found in the [`IRSA TAP documentation page`](https://irsa.ipac.caltech.edu/docs/program_interface/TAP.html).

We first add a cell to the layout that will hold the table; this cell is shown at row = 0, col = 0, with width = 4, height = 2. Once the cell is created, we can request the necessary data from the catalog and display the data as a table using the [`show_table`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_table) method.
```{code-cell} ipython3
BASE_URL = "https://irsa.ipac.caltech.edu/TAP/sync"
QUERY = """
SELECT *
FROM neowiser_p1ba_mch AS n
WHERE n.objid LIKE '(558)%'
"""
table_url = f"{BASE_URL}?QUERY={urllib.parse.quote_plus(QUERY)}"
table_url
```

Now, we can request the necessary data from the catalog and display the data as a table in the Firefly client, using the [`show_table`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_table) method.

Alternatively, we can download data from the catalog using [`astropy.utils.data.download_file`](https://docs.astropy.org/en/stable/api/astropy.utils.data.download_file.html) and upload it to the Firefly client shown in the cell below the first method.

```{code-cell} ipython3
r = fc.add_cell(0, 0, 4, 2, 'tables', 'main')
if r['success']:
table_url = ("https://irsa.ipac.caltech.edu/TAP/sync?QUERY=SELECT+*+FROM+neowiser_p1ba_mch+AS+n+WHERE+n.objid+like+'(558)%'")
fc.show_table(table_url, tbl_id='tableneo', title='558 Carmen NeoWise Catalog', page_size=50)
fc.show_table(table_url, tbl_id='tableneo', title='558 Carmen NeoWise Catalog', page_size=50)
```

```{code-cell} ipython3
# r = fc.add_cell(0, 0, 4, 2, 'tables', 'main')
# tablename = astropy.utils.data.download_file(table_url, timeout=120, cache=True)
# file = fc.upload_file(tablename)
# fc.show_table(file, tbl_id='tableneo', title='558 Carmen Catalog', page_size=50)
```

# if r['success']:
# table_url = ("https://irsa.ipac.caltech.edu/TAP/sync?QUERY=SELECT+*+FROM+neowiser_p1ba_mch+AS+n+WHERE+n.objid+like+'(558)%'")
# tablename = astropy.utils.data.download_file(table_url, timeout=120, cache=True)
Note that along with the table, firefly also displays the coverage and chart associated with the table. It overlays colored squares for each row of the table onto a HiPS image, because the table contains recognizable celestial coordinates. It also creates a scatter plot of ra and dec from the table.

# file = fc.upload_file(tablename)
# fc.show_table(file, tbl_id='tableneo', title='558 Carmen Catalog', page_size=50)
```
+++

## Step 3

After retrieving the data and displaying it in the client, we can now create a light curve by plotting the Modified Julian Date ('mjd') in the abscissa and the magnitude from band W1 ('w1mpro') in the ordinate. We also flip the ordinate to accurately display magnitude.

```{code-cell} ipython3
r = fc.add_cell(2, 0, 2, 2, 'plot-image', 'light-curve')
if r['success']:
status = fc.show_xyplot(tbl_id='tableneo', xCol='mjd', yCol='w1mpro', yOptions='flip')
fc.show_xyplot(tbl_id='tableneo', xCol='mjd', yCol='w1mpro', yOptions='flip')
```

## Step 4

Finally, we can overlay the catalog of data in the table onto a HiPS image using the [`show_coverage`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_coverage) method.

You will notice that there are colored squares that signify where the object was observed based on the RA and Dec given in the catalog.

```{code-cell} ipython3
r = fc.add_cell(2, 2, 2, 2, 'catalog-image', 'target')
if r['success']:
fc.show_coverage();
```

Alternatively, we can queue a HiPS image using the method [`show_hips`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_hips). However, this method requires target coordinates for the object you want to analyze.
Finally, we can overlay the catalog of data in the table onto a HiPS image of our choice, using the method [`show_hips`](https://caltech-ipac.github.io/firefly_client/api/firefly_client.FireflyClient.html#firefly_client.FireflyClient.show_hips). However, this method requires target coordinates for the object you want to analyze.

```{code-cell} ipython3
target='229.851396;-9.720647;EQ_J2000'
viewer_id = 'hipsDiv'
hips_url = 'http://alasky.u-strasbg.fr/AllWISE/RGB-W4-W2-W1'
r = fc.add_cell(2, 2, 2, 2, 'catalog-image', 'target')
if r['success']:
status = fc.show_hips(viewer_id=viewer_id, plot_id='aHipsID1-1', hips_root_url = hips_url,
fc.show_hips(viewer_id=viewer_id, plot_id='aHipsID1-1', hips_root_url = hips_url,
Title='HiPS-WISE', WorldPt=target)
```

Expand All @@ -146,7 +137,7 @@ Firefly allows you to visualize data for specific targets. In conjuction with As

1. We import all necessary modules to create a Firefly client and to download the catalog of data for our target.

2. We start the client in our web browser and prep the layout to appropiately display our tables, plots and images.
2. We start the client in our web browser to appropiately display our tables, plots and images.

3. We use the TAP schema to display the data for our target &mdash; [`558 Carmen`](https://irsa.ipac.caltech.edu/irsaviewer/?__action=table.search&request=%7B%22startIdx%22%3A0%2C%22SearchMethod%22%3A%22AllSky%22%2C%22RequestedDataSet%22%3A%22NEOWISE%20Reactivation%20Database%22%2C%22id%22%3A%22GatorQuery%22%2C%22tbl_id%22%3A%22tbl_id-cf48-45%22%2C%22META_INFO%22%3A%7B%22title%22%3A%22WISE-neowiser_p1ba_mch%20(AllSky)%22%2C%22tbl_id%22%3A%22tbl_id-cf48-45%22%2C%22tbl_pref_key%22%3A%22WISE-neowiser_p1ba_mch%22%7D%2C%22catalogProject%22%3A%22WISE%22%2C%22catalog%22%3A%22neowiser_p1ba_mch%22%2C%22constraints%22%3A%22objid%20like%20%27%25(558)%20Carmen%25%27%22%2C%22pageSize%22%3A100%7D&options=%7B%22backgroundable%22%3Atrue%2C%22pageSize%22%3A100%7D) &mdash; via a table and visualize such data through charts.

Expand All @@ -161,5 +152,5 @@ Firefly allows you to visualize data for specific targets. In conjuction with As
+++

**Author:** Eric Bratton II (IRSA Scientist) in conjunction with the IRSA Science Team<br>
**Updated On:** 2024-07-31<br>
**Updated On:** 2024-10-17<br>
**Contact:** [email protected] or https://irsa.ipac.caltech.edu/docs/help_desk.html
25 changes: 12 additions & 13 deletions tutorials/firefly/SEDs_in_Firefly.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -553,7 +553,7 @@ phot_tbl

This notebook assumes that [jupyter_firefly_extensions](https://github.com/Caltech-IPAC/jupyter_firefly_extensions) has been installed, and that the Firefly server to use has been specified before Jupyterlab was started.

If everything has been properly configured, executing the next cell will bring up a Firefly tab in Jupyterlab with the message "Firefly Ready":
If everything has been properly configured, executing the next cell will bring up a Firefly tab in Jupyterlab with the welcome message.

```{code-cell} ipython3
# Uncomment to use the jupyter_firefly_extensions
Expand All @@ -566,7 +566,6 @@ fc = FireflyClient.make_client(url="https://irsa.ipac.caltech.edu/irsaviewer")
In the event that there are problems with the tab opened above, run the below command to obtain a web link that can be opened in a browser directly:

```{code-cell} ipython3
# Temporary work-around to Firefly error for Slate and/or Internet connection problems
fc.display_url()
```

Expand All @@ -577,7 +576,7 @@ tblpath = "./phot_tbl.fits"
phot_tbl.write(tblpath, format="fits", overwrite=True)
```

Upload the table to Firefly:
Upload the table (FITS file) to Firefly:

```{code-cell} ipython3
tval = fc.upload_file(tblpath)
Expand Down Expand Up @@ -667,9 +666,9 @@ j_fname, h_fname, k_fname = get_2mass_images(target)
```

```{code-cell} ipython3
fc.show_fits(fc.upload_file(j_fname), plot_id="2MASS J")
fc.show_fits(fc.upload_file(h_fname), plot_id="2MASS H")
fc.show_fits(fc.upload_file(k_fname), plot_id="2MASS K")
fc.show_fits(fc.upload_file(j_fname), plot_id="2MASS J", title="2MASS J")
fc.show_fits(fc.upload_file(h_fname), plot_id="2MASS H", title="2MASS H")
fc.show_fits(fc.upload_file(k_fname), plot_id="2MASS K", title="2MASS K")
```

```{code-cell} ipython3
Expand All @@ -684,14 +683,14 @@ fc.set_pan(plot_id="2MASS K", x=target.ra.deg,
y=target.dec.deg, coord="j2000")
```

In the future, the user should be able to implement turning on "Align and Lock by WCS" using the Python API as part of this notebook.
For turning on "Align and Lock by WCS" in these images, there's no dedicated method in Firefly but you can use this workaround (using low-level API):

+++

The current alternative is to use the following link to launch IRSA Finder Chart in a separate browser and turn on that function there:
https://irsa.ipac.caltech.edu/applications/finderchart/servlet/api?mode=getResult&locstr=270.599026+-24.147018+Equ+J2000&subsize=0.0167&sources=2MASS&DoSearch=true
```{code-cell} ipython3
fc.dispatch('ImagePlotCntlr.wcsMatch',
payload=dict(matchType='Standard', lockMatch=True))
```

In that Firefly display, it should be possible to turn on "Align and Lock by WCS" in the image toolbar.
Note that zooming or panning one image will do the same in the other images as well.

+++

Expand All @@ -705,7 +704,7 @@ In that Firefly display, it should be possible to turn on "Align and Lock by WCS

**Author:** David Shupe (IRSA Scientist) and Joyce Kim (IRSA Scientist) in conjunction with the IRSA Science Team

**Updated On:** 2024-07-31
**Updated On:** 2024-10-17

**Contact:** [email protected] or https://irsa.ipac.caltech.edu/docs/help_desk.html

Expand Down

0 comments on commit f3e1bd9

Please sign in to comment.