add CRS Support for OGC API Feature pygeoapi Provider (#1174)

* OGC API - Features Part 2 (groundwork+CRS-BBOX) from PR #1155 - contributes to issue #1128 * #1128 provide conformance class for OAPIF Part 2 in /conformance page * #1128 bitten by flake8... * #1128 configurability CRS Feature Providers with syntax, defaults and tests * #1128 configurability CRS Feature Providers refine for default values * #1128 display supported CRSs in HTML Collection template * #1128 config, mmetadata and tests for storageCRS and storageCrsCoordinateEpoch * #1128 WIP for bbox-crs parameter support * #1128 utility function and tests for default/mandatory supprted CRS list * #1128 default supported CRS adaptation to OAPIF Part 2 standard * #1128 grr flake8 whitespace * #1128 start adding full API tests OGR for bbox-crs and crs parms * #1128 fix flake8 * #1128 fix flake8 - install GDAL in workflow main for OGR tests * #1128 fix flake8 - install GDAL in workflow main for OGR tests - need pip package? * #1128 fix flake8 - install GDAL in workflow main for OGR tests - using libgdal-dev gdal-bin * #1128 fix SensorThings test for main.yml Workflow * #1128 fix SensorThings test for main.yml Workflow nr 2 * #1128 make all OGR tests working again * #1128 make all OGR tests working again - flake8 * #1128 make all OGR tests working again - GeoSolutions WFS bbox * #1128 #1155 add documentation for OGC OAPIF Part 2 CRS CRS BBOX support * #1128 #1155 refine documentation for OGC OAPIF Part 2 CRS CRS BBOX support * #1128 #1155 refine documentation to align with #1149 * #1128 #1155 rework from review OAS and pygeoapi config schema * #1128 #1155 minor: compile Re for CRS URI only once as global var * #1128 merge in changes from PR #1173 - fix missing import * WIP Ogcapi features part 2 - Support for crs query parameter (#1149) * feat(ogcapi_features_crs): start implementing crs support from ogcapi features part2 * Pass input and output CRSs WKT instead of crs transformation object * fix longs lines and blank lines * fix typo * fix import for type annotation not supported by python version * fix variable visibility in local scope * fix tabs/spaces indentations * Add support for the crs parameter to OGRProvider * make flake8 happy * Make crs transformation mechanism more consistent between PostgreSQL and OGR providers * test(util): add two test functions in util.py New functions: test_get_crs_from_uri and test_get_transform_from_crs * fix too long lines... * Update get_crs_from_uri and corresponding test function * fix(get_crs_from_uri): make the error more explicit in if wrong crs uri format * flake8 again... * Keep support for source_srs/target_srs in config for OGRProvider * revert changes made to pygeoapi-config-0.x.yml, overlap with PR 1155 * test: add test data and update test config file * Extract 'crs' and 'storage_crs' and provider level instead of collection level * feat(crs): new decorator to support coordinates transformation of feature collections * feat(crs): 'crs' query parameter for CSVProvider * test(crs): add tests for 'crs' query parameter * test: update number of collections in test_describe_collections * test: update number of collections in test_filter_dict_by_key_value * fix(crs_transform): change the crs transformation decorator Change the logic of the decorator so that it works for both functions that return FeatureCollections and for functions tha return single Features. * test: add tests for get_collection_item end-point with 'crs' parameter * fix(test_get_collection_item_crs): id as path parameter, not query parameter * test: unpack coordinates to create point geometry * feat(crs): add suuport for crs query parameter for all providers of type 'feature' * docs(crs): add documentation to illustrate use of 'crs' query parameters * docs(crs): more data access examples * fix typo and add new line * refactor: specify None as default value for crs_transform_out parameter in _sqlalchemy_to_feature method * changes for PR 1149, test_api and style formatting * CRS84 as default crs also for test_get_collection_items_crs * test(crs): test coordinates transformation implementation of PostgreSQLProvider * test(crs): move tests to test_postgresql_provider * fix test function calls * change test to ensure returned features are the same * add json format to request object * test(crs): test coordinates transformation implementation of OGRProvider * refactor(crs): make more compact get_collection_item and get_collection_items Define two new static methods in API class, to create crs_transform_wkt and setting content-crs header. These methods can be re-used in both get_collection_item and get_collection_items methods and removes code duplication. --------- Co-authored-by: Just van den Broecke <[email protected]> * #1178 fix flake8 error * #1178 use EPSG:28992 i.s.o. 32631 - fix unit test OGR Shapefile * #1174 use CRS-compliant Axis ordering for crs support * #1174 fix and honour CRS 4258disable native CRS Transform in OGR Provider - Axis ordering not honoured... * #1174 remove ADR tests rom test_util.py * #1174 enable native CRS transform again in OGR Provider * #1174 enable native CRS transform again in OGR Provider - fix config * #1174 remove support for source/target_srs in OGRProvider - enforce transforms always based on storageCRS * #1174 fix tests Postgresql Provider for Transforms * #1174 fix tests Postgresql Provider for Transforms * #1174 add tests for OGR Transformation and Axis Order * #1174 Suppress potential axis-swapping in OGR ExportToJSON * #1174 minor fix test - unassign spatialref before setgeom infeat * #1174 minor fix test - unassign spatialref before setgeom infeat - flake8 * #1174 solve CI WFS test failures with GDAL HTTP config options * #1174 bbox and bbox-crs defs local in openapi.py for CITE validators * #1174 merge master - #1152 #1203 etc * #1174 small doc changes * #1174 move GeomObject typedef to beginning of util.py * #1174 added debug logging in transform Decorator func --------- Co-authored-by: Mathieu Tachon <[email protected]>
geopython · Apr 11, 2023 · 7c69937 · 7c69937
1 parent d594653
commit 7c69937
Show file tree

Hide file tree

Showing 40 changed files with 2,516 additions and 698 deletions.
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -80,6 +80,11 @@ jobs:
       with:
         packages: libsqlite3-mod-spatialite
         version: 4.3.0a-6build1
+    - name: Install GDAL with Python bindings
+      uses: awalsh128/cache-apt-pkgs-action@latest
+      with:
+        packages: gdal-bin libgdal-dev
+        version: 3.0.4
     - name: Install requirements 📦
       run: |
         pip3 install -r requirements.txt
@@ -89,6 +94,7 @@ jobs:
         python3 setup.py install
         pip3 install --upgrade numpy elasticsearch
         pip3 install --upgrade numpy "sqlalchemy<2"
+        pip3 install --global-option=build_ext --global-option="-I/usr/include/gdal" GDAL==`gdal-config --version`
         #pip3 install --upgrade rasterio==1.1.8
     - name: setup test data ⚙️
       run: |
@@ -102,6 +108,7 @@ jobs:
         POSTGRESQL_PASSWORD: ${{ secrets.DatabasePassword || 'postgres' }}
       run: |
         pytest tests/test_api.py
+        pytest tests/test_api_ogr_provider.py
         pytest tests/test_config.py
         pytest tests/test_csv__formatter.py
         pytest tests/test_csv__provider.py
@@ -110,12 +117,12 @@ jobs:
         pytest tests/test_filesystem_provider.py
         pytest tests/test_geojson_provider.py
         pytest tests/test_mongo_provider.py
-        #pytest tests/test_ogr_csv_provider.py
-        #pytest tests/test_ogr_esrijson_provider.py
-        #pytest tests/test_ogr_gpkg_provider.py
-        #pytest tests/test_ogr_shapefile_provider.py
-        #pytest tests/test_ogr_sqlite_provider.py
-        #pytest tests/test_ogr_wfs_provider.py
+        pytest tests/test_ogr_csv_provider.py
+        pytest tests/test_ogr_esrijson_provider.py
+        pytest tests/test_ogr_gpkg_provider.py
+        pytest tests/test_ogr_shapefile_provider.py
+        pytest tests/test_ogr_sqlite_provider.py
+        pytest tests/test_ogr_wfs_provider.py
         pytest tests/test_openapi.py
         pytest tests/test_postgresql_provider.py
         pytest tests/test_rasterio_provider.py

diff --git a/.gitignore b/.gitignore
@@ -115,3 +115,9 @@ ENV/
 
 .DS_Store
 examples/django/sample_project/db.sqlite3
+
+# Backup files
+*~
+
+# Pre-commit hooks config file
+.pre-commit-config.yaml
diff --git a/aws-lambda/function/pygeoapi-config.yml b/aws-lambda/function/pygeoapi-config.yml
@@ -296,8 +296,8 @@ resources:
               data:
                   source_type: WFS
                   source: WFS:https://service.pdok.nl/kadaster/rdinfo/wfs/v1_0?
-                  source_srs: EPSG:28992
-                  target_srs: EPSG:4326
+#                  source_srs: EPSG:28992
+#                  target_srs: EPSG:4326
                   source_capabilities:
                       paging: True
 
@@ -312,6 +312,12 @@ resources:
                       # GDAL_PROXY_AUTH: (optional auth for remote WFS)
                       CPL_DEBUG: NO
 
+              crs:
+                - http://www.opengis.net/def/crs/OGC/1.3/CRS84
+                - http://www.opengis.net/def/crs/EPSG/0/4326
+                - http://www.opengis.net/def/crs/EPSG/0/4258
+                - http://www.opengis.net/def/crs/EPSG/0/28992
+              storage_crs: http://www.opengis.net/def/crs/EPSG/0/28992
               id_field: gml_id
               layer: rdinfo:stations
 

diff --git a/docker/default.config.yml b/docker/default.config.yml
@@ -211,8 +211,8 @@ resources:
               data:
                   source_type: WFS
                   source: WFS:https://service.pdok.nl/kadaster/rdinfo/wfs/v1_0?
-                  source_srs: EPSG:28992
-                  target_srs: EPSG:4326
+#                  source_srs: EPSG:28992
+#                  target_srs: EPSG:4326
                   source_capabilities:
                       paging: True
 
@@ -227,6 +227,10 @@ resources:
                       # GDAL_PROXY_AUTH: (optional auth for remote WFS)
                       CPL_DEBUG: NO
 
+              crs:
+                - http://www.opengis.net/def/crs/EPSG/0/4258
+                - http://www.opengis.net/def/crs/EPSG/0/28992
+              storage_crs: http://www.opengis.net/def/crs/EPSG/0/28992
               id_field: gml_id
               layer: rdinfo:stations
 
@@ -258,8 +262,8 @@ resources:
               data:
                   source_type: WFS
                   source: WFS:http://demo.deegree.org/utah-workspace/services/wfs?TYPENAME=app:SGID93_LOCATION_UDOTMap_CityLocations
-                  source_srs: EPSG:26912
-                  target_srs: EPSG:4326
+#                  source_srs: EPSG:26912
+#                  target_srs: EPSG:4326
                   source_capabilities:
                       paging: True
 
@@ -274,6 +278,10 @@ resources:
                       # GDAL_PROXY_AUTH: (optional auth for remote WFS)
                       CPL_DEBUG: NO
 
+              crs:
+                - http://www.opengis.net/def/crs/EPSG/0/4258
+                - http://www.opengis.net/def/crs/EPSG/0/26912
+              storage_crs: http://www.opengis.net/def/crs/EPSG/0/26912
               id_field: NAME
               layer: app:SGID93_LOCATION_UDOTMap_CityLocations
 
@@ -304,8 +312,8 @@ resources:
               data:
                   source_type: WFS
                   source: WFS:https://gs-stable.geosolutionsgroup.com/geoserver/wfs
-                  source_srs: EPSG:32632
-                  target_srs: EPSG:4326
+#                  source_srs: EPSG:32632
+#                  target_srs: EPSG:4326
                   source_capabilities:
                       paging: True
 
@@ -320,6 +328,10 @@ resources:
                       # GDAL_PROXY_AUTH: (optional auth for remote WFS)
                       CPL_DEBUG: NO
 
+              crs:
+                - http://www.opengis.net/def/crs/EPSG/0/4258
+                - http://www.opengis.net/def/crs/EPSG/0/32632
+              storage_crs: http://www.opengis.net/def/crs/EPSG/0/32632
               id_field: gml_id
               layer: unesco:Unesco_point
 
@@ -355,8 +367,8 @@ resources:
               data:
                   source_type: GPKG
                   source: tests/data/poi_portugal.gpkg
-                  source_srs: EPSG:4326
-                  target_srs: EPSG:4326
+#                  source_srs: EPSG:4326
+#                  target_srs: EPSG:4326
                   source_capabilities:
                       paging: True
 
@@ -395,8 +407,8 @@ resources:
               data:
                   source_type: GeoJSON
                   source: tests/data/ne_110m_lakes.geojson
-                  source_srs: EPSG:4326
-                  target_srs: EPSG:4326
+#                  source_srs: EPSG:4326
+#                  target_srs: EPSG:4326
                   source_capabilities:
                       paging: True
 
@@ -438,8 +450,8 @@ resources:
                   source_type: SQLite
                   # source: tests/data/ne_110m_admin_0_countries.sqlite
                   source: tests/data/dutch_addresses_4326.sqlite
-                  source_srs: EPSG:4326
-                  target_srs: EPSG:4326
+#                  source_srs: EPSG:4326
+#                  target_srs: EPSG:4326
                   source_capabilities:
                       paging: True
 
@@ -480,8 +492,8 @@ resources:
               data:
                   source_type: GPKG
                   source: tests/data/dutch_addresses_4326.gpkg
-                  source_srs: EPSG:4326
-                  target_srs: EPSG:4326
+#                  source_srs: EPSG:4326
+#                  target_srs: EPSG:4326
                   source_capabilities:
                       paging: True
 

diff --git a/docs/source/configuration.rst b/docs/source/configuration.rst
@@ -161,8 +161,8 @@ default.
               - observations
               - monitoring
           linked-data: # linked data configuration (see Linked Data section)
-              item_template: tests/data/base.jsonld 
-              context:  
+              item_template: tests/data/base.jsonld
+              context:
                   - datetime: https://schema.org/DateTime
                   - vocab: https://example.com/vocab#
                     stn_id: "vocab:stn_id"
@@ -195,6 +195,13 @@ default.
                 uri_field: uri # optional field corresponding to the Uniform Resource Identifier (see Linked Data section)
                 time_field: datetimestamp  # optional field corresponding to the temporal property of the dataset
                 title_field: foo # optional field of which property to display as title/label on HTML pages
+                crs: # optional: supported CRSs for parameters 'crs' and 'bbox-crs' (OGC OAPIF Part 2)
+                     # default: http://www.opengis.net/def/crs/OGC/1.3/CRS84
+                    - http://www.opengis.net/def/crs/EPSG/0/4326
+                    - http://www.opengis.net/def/crs/EPSG/0/3857
+                    - http://www.opengis.net/def/crs/EPSG/0/28992
+                storage_crs: http://www.opengis.net/def/crs/OGC/1.3/CRS84 # optional CRS in which data is stored, default: as 'crs' field
+                storage_crs_coordinate_epoch: : 2017.23 # optional, if storage_crs is a dynamic coordinate reference system
                 format:  # optional default format
                     name: GeoJSON  # required: format name
                     mimetype: application/json  # required: format mimetype
@@ -203,6 +210,14 @@ default.
                 properties:  # optional: only return the following properties, in order
                     - stn_id
                     - value
+                # coordinate reference systems (CRS) section is optional
+                # default CRSs are http://www.opengis.net/def/crs/OGC/1.3/CRS84 (coordinates without height)
+                # and http://www.opengis.net/def/crs/OGC/1.3/CRS84h (coordinates with ellipsoidal height)
+                storage_crs: http://www.opengis.net/def/crs/EPSG/0/28992 # CRS of the dataset to publish
+                crs: # supported coordinate reference systems (CRS) for 'crs' query parameter
+                    - http://www.opengis.net/def/crs/EPSG/0/28992
+                    - http://www.opengis.net/def/crs/OGC/1.3/CRS84
+                    - http://www.opengis.net/def/crs/EPSG/0/4326
 
       hello-world:  # name of process
           type: collection  # REQUIRED (collection, process, or stac-collection)
@@ -570,12 +585,12 @@ deployment flexibility, the path can be specified with string interpolation of e
 .. code-block:: yaml
 
     linked-data:
-      item_template: tests/data/base.jsonld 
+      item_template: tests/data/base.jsonld
       context:
         - datetime: https://schema.org/DateTime
 
 .. note::
-   The template ``tests/data/base.jsonld`` renders the unmodified JSON-LD. For more information on the capacities 
+   The template ``tests/data/base.jsonld`` renders the unmodified JSON-LD. For more information on the capacities
    of Jinja2 templates, see :ref:`html-templating`.
 
 Summary