diff --git a/docs/tutorials.rst b/docs/tutorials.rst index 9b7217019..fdf0a0806 100644 --- a/docs/tutorials.rst +++ b/docs/tutorials.rst @@ -26,3 +26,12 @@ How to create STAC Catalogs with PySTAC - :ref:`Docs version ` This was a tutorial that was part of a 30 minute presentation at the `community STAC sprint `_ in Arlington, VA in November 2019. It runs through creating a STAC of image or label items from the `SpaceNet 5 `_ dataset. + +Adding New and Custom Extensions +-------------------------------- + +- :tutorial:`GitHub version ` +- :ref:`Docs version ` + +This tutorial goes over how to contribute new extensions to PySTAC as well as how to implement +your own custom extensions. diff --git a/docs/tutorials/adding-new-and-custom-extensions.ipynb b/docs/tutorials/adding-new-and-custom-extensions.ipynb new file mode 100644 index 000000000..1535bb87c --- /dev/null +++ b/docs/tutorials/adding-new-and-custom-extensions.ipynb @@ -0,0 +1,542 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Adding New and Custom Extensions\n", + "\n", + "This tutorial will cover how to add new extensions to PySTAC. It will go over how to contribute a common extension (one found in the [stac-spec repo](https://github.com/radiantearth/stac-spec/tree/v1.0.0-beta.2/extensions)), as well as how to register a custom extension with PySTAC.\n", + "\n", + "We'll work on implementing the [Satellite Extension](https://github.com/radiantearth/stac-spec/tree/v1.0.0-beta.2/extensions/sat) with a modified extension ID, registering it as `space_camera` instead of `sat`. " + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import pystac" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If we were implementing the `sat` extension for real, we would make sure there is an entry for our extension in the `pystac.extensions.Extensions` object [found here](https://github.com/azavea/pystac/blob/v0.5.1/pystac/extensions/__init__.py#L10-L27) with the relevant entry. Here we'll just use our own `Extensions` class to define our fake `SPACE_CAMERA` extension ID:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "class Extensions:\n", + " SPACE_CAMERA = 'space_camera'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For this tutorial we'll use some code below to read in an item and modify the real `sat` extension ID into our tutorial `space_camera` ID. If we didn't need to do this modification, we could simply read in the item from the URI using `pystac.read_file`." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "\n", + "def modify_sat_extension_id(item_json):\n", + " item_json['stac_extensions'].remove(pystac.extensions.Extensions.SAT)\n", + " item_json['stac_extensions'].append(Extensions.SPACE_CAMERA)\n", + " \n", + "def read_item(href):\n", + " item_json = json.loads(pystac.STAC_IO.read_text(href))\n", + " modify_sat_extension_id(item_json)\n", + " return pystac.read_dict(item_json)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here we read in an item that implements the `sat` extension, which based on the above code will modify to implement the `space_camera` extension:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "item_before = read_item('https://raw.githubusercontent.com/radiantearth/stac-spec/v1.0.0-beta.2/extensions/sat/examples/example-landsat8.json')\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "item_before.ext.implements(Extensions.SPACE_CAMERA)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Even though the item reports it implements that extension, that extension isn't registered with PySTAC and if we try to access the extension functionality it will tell us so:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "ename": "ExtensionError", + "evalue": "'space_camera' is not an extension registered with PySTAC", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mExtensionError\u001b[0m Traceback (most recent call last)", + "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mitem_before\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mext\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mExtensions\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSPACE_CAMERA\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[0;32m~/proj/stac/pystac/venv/lib/python3.6/site-packages/pystac-0.5.0-py3.6.egg/pystac/stac_object.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, extension_id)\u001b[0m\n\u001b[1;32m 39\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mpystac\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSTAC_EXTENSIONS\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_registered_extension\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mextension_id\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 40\u001b[0m raise ExtensionError(\"'{}' is not an extension \"\n\u001b[0;32m---> 41\u001b[0;31m \"registered with PySTAC\".format(extension_id))\n\u001b[0m\u001b[1;32m 42\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 43\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimplements\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mextension_id\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", + "\u001b[0;31mExtensionError\u001b[0m: 'space_camera' is not an extension registered with PySTAC" + ] + } + ], + "source": [ + "item_before.ext[Extensions.SPACE_CAMERA]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So let's implement it!\n", + "\n", + "### Implementing an ItemExtension\n", + "\n", + "We'll be referring to the [Satellite Extensions Specification](https://github.com/radiantearth/stac-spec/blob/v1.0.0-beta.2/extensions/sat/README.md) (refered to as the spec) to implement this extension.\n", + "\n", + "The `sat` extension (or in our case `space_camera` extension) is scoped to `Item`. That information is found in the \"Scope\" line at the top of the spec. We'll want to implement an `CatalogExtension`, `CollectionExtension`, and `ItemExtension` for each of the STAC object types in the scope. In this case, we're only implementing an `ItemExtension`." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "from pystac.extensions.base import ItemExtension" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To implement the object extension, create a child class that implements each of the abstract methods of the relevant base class. For `ItemExtension`, the only required methods are listed below, along with an appropriate `__init__` method:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "class SatItemExt(ItemExtension):\n", + " def __init__(self, item):\n", + " self.item = item\n", + " \n", + " @classmethod\n", + " def from_item(self, item):\n", + " return SatItemExt(item)\n", + "\n", + " @classmethod\n", + " def _object_links(cls):\n", + " return []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `from_item` class method simply returns a new instance of the item extension given an item.\n", + "\n", + "The `_object_links` class method returns the `rel` string for any links that point to STAC objects like Catalogs, Collections or Items. PySTAC needs to know which links point to STAC objects because it needs to consider them when fully resolving a STAC into in-memory objects. In a lot of cases, extensions don't add new links to STAC objects, so this is normally an empty list; however, if the extension does do this (like the `source` link in the [Label Extension](https://github.com/radiantearth/stac-spec/tree/v1.0.0-beta.2/extensions/label#links-source-imagery)), make sure to return the correct value (like the LabelItemExt is doing [here](https://github.com/azavea/pystac/blob/v0.5.0/pystac/extensions/label.py#L291-L293))." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Defining properties\n", + "\n", + "An extension object works by modifying the Item (or whichever STAC object is being extended) directly through Python [property getters and setters](https://docs.python.org/3/library/functions.html#property). The getter should read directly from the `properties` or `extra_fields` in the item and perform any trasformations needed to convert to the relevant Python objects (e.g. transform a string into a `datetime` object). Likewise, the setter should take in Python objects and transform them to their serialized string, and set them in the appropriate place in item. This way the extension modifies the Item directly, and will not require any specialized serialization or deserialization logic. This also allows multiple extensions to be used to access and set information on the STAC object - a distinct advantage to the inheritance-based extension implementation that PySTAC used before 0.4.0.\n", + "\n", + "For the `sat` extension we have two properties to implement, both of which are straighforward and do not need any transformation in the getters and setters:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "class SatItemExt(ItemExtension):\n", + " def __init__(self, item):\n", + " self.item = item\n", + " \n", + " @property\n", + " def orbit_state(self):\n", + " \"\"\"\"ADD DOCSTRING!\"\"\"\n", + " return self.item.properties.get('sat:orbit_state')\n", + " \n", + " @orbit_state.setter\n", + " def orbit_state(self, v):\n", + " self.item.properties['sat:orbit_state'] = v\n", + " \n", + " @property\n", + " def relative_orbit(self):\n", + " \"\"\"\"ADD DOCSTRING!\"\"\"\n", + " return self.item.properties.get('sat:relative_orbit')\n", + " \n", + " @relative_orbit.setter\n", + " def relative_orbit(self, v):\n", + " self.item.properties['sat:relative_orbit'] = v\n", + " \n", + " @classmethod\n", + " def from_item(self, item):\n", + " return SatItemExt(item)\n", + "\n", + " @classmethod\n", + " def _object_links(cls):\n", + " return []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Extensions also define an `apply` method that encodes the optional and required values that go into the extension. The `apply` should list all of the values of the extension, and give default values of `None` to optional parameters. That way a users adding an extension to an object can easily tell what values are needed to implement the extension.\n", + "\n", + "Here we use the `apply` method to encode the requirement in the spec that at least one of `orbit_state` and `relative_orbit` need to be defined:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "class SatItemExt(ItemExtension):\n", + " def __init__(self, item):\n", + " self.item = item\n", + " \n", + " def apply(self, orbit_state=None, relative_orbit=None):\n", + " \"\"\"Applies Satellite extension properties to the extended Item.\n", + " \n", + " Args:\n", + " orbit_state (str): The state of the orbit. Either ascending or descending \n", + " for polar orbiting satellites, or geostationary for geosynchronous satellites\n", + " relative_orbit (int): The relative orbit number at the time of acquisition.\n", + " \n", + " Note:\n", + " At least one property must be supplied.\n", + " \"\"\"\n", + " if orbit_state is None and relative_orbit is None:\n", + " raise pystac.STACError(\"sat extension needs at least one property value.\")\n", + " \n", + " self.orbit_state = orbit_state\n", + " self.relative_orbit = relative_orbit\n", + " \n", + " @property\n", + " def orbit_state(self):\n", + " return self.item.properties.get('sat:orbit_state')\n", + " \n", + " @orbit_state.setter\n", + " def orbit_state(self, v):\n", + " self.item.properties['sat:orbit_state'] = v\n", + " \n", + " @property\n", + " def relative_orbit(self):\n", + " return self.item.properties.get('sat:relative_orbit')\n", + " \n", + " @relative_orbit.setter\n", + " def relative_orbit(self, v):\n", + " self.item.properties['sat:relative_orbit'] = v\n", + " \n", + " @classmethod\n", + " def from_item(self, item):\n", + " return SatItemExt(item)\n", + "\n", + " @classmethod\n", + " def _object_links(cls):\n", + " return []" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we have our object extension we need to register it with PySTAC. To do so we'll need to define an `ExtendedObject` to tie together the PySTAC object we are extending and our `SatItemExt` class:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "from pystac.extensions.base import ExtendedObject" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "extended_object = ExtendedObject(pystac.Item, SatItemExt)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then we define an `ExtensionDefinition` that ties together our extension ID with the list of object extensions. In this case, we are only extending Item and so there's only a single entry in the list:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "from pystac.extensions.base import ExtensionDefinition" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "extension_definition = ExtensionDefinition(Extensions.SPACE_CAMERA, [extended_object])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For common extensions this definition usually happens at the end of the extension file all in one line; see [this example](https://github.com/azavea/pystac/blob/v0.5.0/pystac/extensions/label.py#L656-L657)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can register the extension definition with PySTAC. For common extensions defined in the library [you would add it to the list in the top level package __init__](https://github.com/azavea/pystac/blob/v0.5.1/pystac/__init__.py#L32-L43). However if you're creating a custom extension you can use the following method:" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [], + "source": [ + "pystac.STAC_EXTENSIONS.add_extension(extension_definition)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Remember, if you are implementing an extension in PySTAC, make sure to add thorough unit tests ([example](https://github.com/azavea/pystac/blob/v0.5.0/tests/extensions/test_view.py)) and add the extension to the documentation ([example](https://github.com/azavea/pystac/blob/v0.5.0/docs/api.rst#view-geometry-extension))!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Using the extension\n", + "\n", + "When we read the item (again manipulating the JSON so that the `sat` extension ID turns into `space_camera`), we can now access the extension functionality through the same means as the other extensions:" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "item_after = read_item('https://raw.githubusercontent.com/radiantearth/stac-spec/v1.0.0-beta.2/extensions/sat/examples/example-landsat8.json')" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'ascending'" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "item_after.ext[Extensions.SPACE_CAMERA].orbit_state" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "item_after.ext.space_camera.relative_orbit = 5" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notice that setting the property value through the extension sets the correct item property:" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "5" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "item_after.properties['sat:relative_orbit']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also read in an item that does not already implement the extension, enable it, and use the `apply` method to fill out the values:" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [], + "source": [ + "item3 = pystac.read_file('https://raw.githubusercontent.com/radiantearth/stac-spec/v1.0.0-beta.2/item-spec/examples/sample-full.json')" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "item3.ext.enable(Extensions.SPACE_CAMERA)" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Help on method apply in module __main__:\n", + "\n", + "apply(orbit_state=None, relative_orbit=None) method of __main__.SatItemExt instance\n", + " Applies Satellite extension properties to the extended Item.\n", + " \n", + " Args:\n", + " orbit_state (str): The state of the orbit. Either ascending or descending \n", + " for polar orbiting satellites, or geostationary for geosynchronous satellites\n", + " relative_orbit (int): The relative orbit number at the time of acquisition.\n", + " \n", + " Note:\n", + " At least one property must be supplied.\n", + "\n" + ] + } + ], + "source": [ + "help(item3.ext.space_camera.apply)" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "item3.ext.space_camera.apply(relative_orbit='ascending')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.5" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}