Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactors architecture to better enable STAC 1.0 extension changes #309

Merged
merged 54 commits into from
May 6, 2021
Merged
Show file tree
Hide file tree
Changes from 46 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
d71c303
Add test files for 1.0.0-RC1
lossyrob Mar 11, 2021
d531cc2
Add type annotations to everything
lossyrob Apr 22, 2021
4bb702a
Update code and tests to pass with type annotations
lossyrob Apr 25, 2021
c78f832
Add mypy to test step; fixes based on mypy errors
lossyrob Apr 25, 2021
73fad0d
Remove LinkMixin, implement directly into STACObject
lossyrob Apr 25, 2021
4100dae
Use underscore naming convention for TYPE_CHECKING imports
lossyrob Apr 25, 2021
330e1b7
New extension architecture in place, refactor eo, file and label.
lossyrob Apr 26, 2021
a72413a
Implement collection assets; extension summaries; more fixes.
lossyrob Apr 27, 2021
e968349
Rename examples with consistent version tag.
lossyrob Apr 27, 2021
8b873ad
Modify change_stac_version to perform migration
lossyrob Apr 27, 2021
1b93500
Set schema_uri during eo migration
lossyrob Apr 27, 2021
421788f
More extension refactors
lossyrob Apr 28, 2021
8a2de1d
Merge remote-tracking branch 'origin/main' into feature/rde/1.0-refac…
lossyrob Apr 28, 2021
f70e364
Remove single-file-stac extensions
lossyrob Apr 28, 2021
aa57ae5
More extension refactors
lossyrob Apr 28, 2021
a1f5096
All extensions refactored to new architecture
lossyrob Apr 29, 2021
245a6db
Fixed identify tests; fixing migration teses
lossyrob Apr 29, 2021
d614036
Update version schema to fixed published version
lossyrob Apr 29, 2021
dc80bb5
Migrate checksum -> file
lossyrob Apr 29, 2021
77ea376
Remove collection-assets extension
lossyrob Apr 29, 2021
aa98fde
Update STAC version to 1.0.0-rc.3
lossyrob Apr 29, 2021
9b8e1bc
Set href during migrate
lossyrob Apr 29, 2021
f566aa9
Change extension API to use static method on extension class
lossyrob Apr 30, 2021
f69b305
Migrate test files to 1.0.0-rc.3
lossyrob Apr 30, 2021
7438ee8
Add datacube extension
lossyrob Apr 30, 2021
1e0ae07
Add item_assets extension
lossyrob Apr 30, 2021
aba25aa
Fixe tests
lossyrob Apr 30, 2021
71cba8a
yapf formatting
lossyrob Apr 30, 2021
c073712
Switch to pyright
lossyrob Apr 30, 2021
b2e2441
Flake8 fixes
lossyrob Apr 30, 2021
f6e1716
Remove dataclasses usage as we're still supporting python 3.6
lossyrob Apr 30, 2021
d0c6cc5
Refactor STAC_IO to StacIO
lossyrob Apr 30, 2021
501e98e
Move to black as formatter
lossyrob Apr 30, 2021
aa57019
Reformat with black
lossyrob Apr 30, 2021
a1019e6
Modify flake8 config to match black
lossyrob Apr 30, 2021
3f99821
Fixup linting changes with shorter line length requirement
lossyrob May 1, 2021
07f5cc6
Bump sphinx version
duckontheweb May 2, 2021
a45d5e6
Fix basic sphinx build issues
duckontheweb May 2, 2021
eccdf8c
Add note about Sphinx warnings
duckontheweb May 2, 2021
773a571
Remove import pystac as ps in favor of import pystac
lossyrob May 3, 2021
d3e573b
Move extension exceptions into pystac.errors
lossyrob May 3, 2021
f22b587
Move STACValidationError to pystac.errors
lossyrob May 3, 2021
c53428b
Merge remote-tracking branch 'origin/feature/jd/1.0-refactors-doc-fix…
lossyrob May 3, 2021
aa1cdf8
Issue deprecation warnings for STAC_IO
lossyrob May 3, 2021
5680583
Fix stray docstring escape char breaking CI
lossyrob May 3, 2021
af2a6e4
Fix typo
lossyrob May 3, 2021
3e21956
Use Union[Catalog, Collection] as child type in Catalog.
lossyrob May 4, 2021
6236686
Allow all `from_file` overrides to set a stac_io implementation
lossyrob May 4, 2021
20afd2c
Asset.get_absoulte_href returns None if no abs href possible
lossyrob May 4, 2021
9d1ffee
Add test files for 1.0.0-RC3
lossyrob Mar 11, 2021
4e99310
Ensure stac_extensions is a list for consistency
lossyrob May 4, 2021
c71a716
Add 1.0.0-RC3 examples
lossyrob May 4, 2021
f6ee841
Update CHANGELOG
lossyrob May 4, 2021
63f83f4
Merge remote-tracking branch 'origin/main' into feature/rde/1.0-refac…
lossyrob May 5, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
15 changes: 2 additions & 13 deletions .flake8
Original file line number Diff line number Diff line change
@@ -1,14 +1,3 @@
[flake8]
max-line-length = 100

## IGNORES

# E126: yapf conflicts with "continuation line over-indented for hanging indent"

# E127: flake8 reporting incorrect continuation line indent errors
# on multi-line and multi-level indents

# W503: flake8 reports this as incorrect, and scripts/format_code
# changes code to it, so let format_code win.

ignore = E126,E127,W503
max-line-length = 88
extend-ignore = E203, W503, E731, E722
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,14 @@ To build and develop the documentation locally, make sure sphinx is available (w
> make livehtml
```

> Note: You will see some warnings along the lines of
> ```
> WARNING: duplicate object description of pystac.Collection.id,
> other instance in api, use :noindex: for one of them
> ```
> for some of the
> classes. This is expected due to [sphinx-doc/sphinx#8664](https://github.com/sphinx-doc/sphinx/issues/8664).

Use 'make' without arguments to see a list of available commands.

__Note__: `nbsphinx` requires that a local `pystac` is installed; use `pip install -e .`.
Expand Down
162 changes: 92 additions & 70 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -127,13 +127,6 @@ Link
:members:
:undoc-members:

LinkType
~~~~~~~~

.. autoclass:: pystac.LinkType
:members:
:undoc-members:

MediaType
~~~~~~~~~

Expand All @@ -149,7 +142,7 @@ STAC_IO

STAC_IO is the utility mechanism that PySTAC uses for reading and writing. Users of PySTAC can hook into PySTAC by overriding members to utilize their own IO methods.

.. autoclass:: pystac.STAC_IO
.. autoclass:: pystac.stac_io.STAC_IO
:members:
:undoc-members:

Expand Down Expand Up @@ -193,17 +186,20 @@ ExtensionError
Extensions
----------

.. autoclass:: pystac.extensions.Extensions
:members:
:undoc-members:
**TEMPORARILY REMOVED**
.. .. autoclass:: pystac.extensions.Extensions
.. :members:
.. :undoc-members:

ExtensionIndex
~~~~~~~~~~~~~~

An ExtensionIndex is accessed through the :attr:`STACObject.ext <pystac.STACObject.ext>` property and is the primary way to access information and functionality around STAC extensions.
**TEMPORARILY REMOVED**

.. An ExtensionIndex is accessed through the :attr:`STACObject.ext <pystac.STACObject.ext>` property and is the primary way to access information and functionality around STAC extensions.

.. autoclass:: pystac.stac_object.ExtensionIndex
:members: __getitem__, __getattr__, enable, implements
.. .. autoclass:: pystac.stac_object.ExtensionIndex
.. :members: __getitem__, __getattr__, enable, implements


EO Extension
Expand All @@ -214,17 +210,21 @@ These classes are representations of the `EO Extension Spec <https://github.com/
EOItemExt
~~~~~~~~~

.. autoclass:: pystac.extensions.eo.EOItemExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.eo.EOItemExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

Band
~~~~

.. autoclass:: pystac.extensions.eo.Band
:members:
:undoc-members:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.eo.Band
.. :members:
.. :undoc-members:


Label Extension
Expand All @@ -235,10 +235,12 @@ These classes are representations of the `Label Extension Spec <https://github.c
LabelItemExt
~~~~~~~~~~~~

.. autoclass:: pystac.extensions.label.LabelItemExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.label.LabelItemExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

LabelType
~~~~~~~~~
Expand Down Expand Up @@ -283,10 +285,12 @@ Implements the `Point Cloud Extension <https://github.com/radiantearth/stac-spec
PointcloudItemExt
~~~~~~~~~~~~~~~~~

.. autoclass:: pystac.extensions.pointcloud.PointcloudItemExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.pointcloud.PointcloudItemExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

Projection Extension
--------------------
Expand All @@ -296,10 +300,12 @@ Implements the `Projection Extension <https://github.com/radiantearth/stac-spec/
ProjectionItemExt
~~~~~~~~~~~~~~~~~

.. autoclass:: pystac.extensions.projection.ProjectionItemExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.projection.ProjectionItemExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

Timestamps Extension
--------------------
Expand All @@ -309,10 +315,12 @@ Implements the `Timestamps Extension <https://github.com/radiantearth/stac-spec/
TimestampsItemExt
~~~~~~~~~~~~~~~~~

.. autoclass:: pystac.extensions.timestamps.TimestampsItemExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.timestamps.TimestampsItemExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

SAR Extension
-------------
Expand All @@ -322,10 +330,12 @@ Implements the `SAR Extension <https://github.com/radiantearth/stac-spec/tree/v1
SarItemExt
~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: pystac.extensions.sar.SarItemExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.sar.SarItemExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

SAT Extension
-------------
Expand All @@ -335,25 +345,31 @@ Implements the `SAT Extension <https://github.com/radiantearth/stac-spec/tree/v1
SatItemExt
~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: pystac.extensions.sar.SatItemExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.sar.SatItemExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

Single File STAC Extension
--------------------------

These classes are representations of the `Single File STAC Extension <https://github.com/radiantearth/stac-spec/tree/v1.0.0-beta.2/extensions/single-file-stac>`_.

.. automodule:: pystac.extensions.single_file_stac
:members: create_single_file_stac
**TEMPORARILY REMOVED**

.. .. automodule:: pystac.extensions.single_file_stac
.. :members: create_single_file_stac

SingleFileSTACCatalogExt
~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: pystac.extensions.single_file_stac.SingleFileSTACCatalogExt
:members:
:undoc-members:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.single_file_stac.SingleFileSTACCatalogExt
.. :members:
.. :undoc-members:

Version Extension
-----------------
Expand All @@ -363,18 +379,22 @@ Implements the `Version Extension <https://github.com/radiantearth/stac-spec/tre
VersionCollectionExt
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: pystac.extensions.version.VersionCollectionExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.version.VersionCollectionExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

VersionItemExt
~~~~~~~~~~~~~~

.. autoclass:: pystac.extensions.version.VersionItemExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.version.VersionItemExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

View Geometry Extension
-----------------------
Expand All @@ -384,10 +404,12 @@ Implements the `View Geometry Extension <https://github.com/radiantearth/stac-sp
ViewItemExt
~~~~~~~~~~~

.. autoclass:: pystac.extensions.view.ViewItemExt
:members:
:undoc-members:
:show-inheritance:
**TEMPORARILY REMOVED**

.. .. autoclass:: pystac.extensions.view.ViewItemExt
.. :members:
.. :undoc-members:
.. :show-inheritance:

Serialization
-------------
Expand All @@ -401,17 +423,17 @@ Identification
.. automodule:: pystac.serialization
:members: identify_stac_object

.. autoclass:: pystac.serialization.STACJSONDescription
:members:
:undoc-members:
.. .. autoclass:: pystac.serialization.STACJSONDescription
.. :members:
.. :undoc-members:

.. autoclass:: pystac.serialization.STACVersionRange
:members:
:undoc-members:

.. autoclass:: pystac.serialization.STACVersionID
:members:
:undoc-members:
.. .. autoclass:: pystac.serialization.STACVersionID
.. :members:
.. :undoc-members:


Migration
Expand Down Expand Up @@ -456,8 +478,8 @@ schema URIs. A default implementation is included that uses known locations; how
can provide their own schema URI maps in a :class:`~pystac.validation.JsonSchemaSTACValidator`
to modify the URIs used.

.. autoclass:: pystac.validation.SchemaUriMap
:members:
.. .. autoclass:: pystac.validation.SchemaUriMap
.. :members:

.. autoclass:: pystac.validation.schema_uri_map.DefaultSchemaUriMap
:members:
Expand Down
4 changes: 2 additions & 2 deletions docs/concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ are relative to the linking object's file location. For example, if a catalog at
then that link should resolve to the absolute path ``/some/location/item-id/item-id.json``.

Links are set as absolute or relative HREFs at save time, as determine by the root catalog's catalog_type
:attribute:`~pystac.Catalog.catalog_type`. This means that, even if the stored HREF of the link is absolute,
:attr:`~pystac.Catalog.catalog_type`. This means that, even if the stored HREF of the link is absolute,
if the root ``catalog_type=CatalogType.RELATIVE_PUBLISHED`` or ``catalog_type=CatalogType.SELF_CONTAINED``
and subsequent serializing of the any links in the catalog will produce a relative link,
based on the self link of the parent object.
Expand Down Expand Up @@ -396,7 +396,7 @@ Item Asset properties

Properties that apply to Items can be found in two places: the Item's properties or in any of an Item's Assets. If the property is on an Asset, it applies only that specific asset. For example, gsd defined for an Item represents the best Ground Sample Distance (resolution) for the data within the Item. However, some assets may be lower resolution and thus have a higher gsd. In that case, the `gsd` can be found on the Asset.

See the STAC documentation on `Additional Fields for Assets <https://github.com/radiantearth/stac-spec/blob/v1.0.0-beta.2/item-spec/item-spec.md#additional-fields-for-assets>`_ and the relevant `Best Practices <https://github.com/radiantearth/stac-spec/blob/v1.0.0-beta.2/best-practices.md#common-use-cases-of-additional-fields-for-assets>`_ for more information.
See the STAC documentation on `Additional Fields for Assets <https://github.com/radiantearth/stac-spec/blob/v1.0.0-beta.2/item-spec/item-spec.md#additional-fields-for-assets>`_ and the relevant `Best Practices <https://github.com/radiantearth/stac-spec/blob/v1.0.0-beta.2/best-practices.md#common-use-cases-of-additional-fields-for-assets>`__ for more information.

The implementation of this feature in PySTAC uses the method described here and is consistent across Item and ItemExtensions. The bare property names represent values for the Item only, but for each property where it is possible to set on both the Item or the Asset there is a ``get_`` and ``set_`` methods that optionally take an Asset. For the ``get_`` methods, if the property is found on the Asset, the Asset's value is used; otherwise the Item's value will be used. For the ``set_`` method, if an Asset is passed in the value will be applied to the Asset and not the Item.

Expand Down
3 changes: 3 additions & 0 deletions mypy.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[mypy]
ignore_missing_imports = True
disallow_untyped_defs = True
Loading