Skip to content

Commit

Permalink
Issue #720/#402/#725 add documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
soxofaan committed Feb 12, 2025
1 parent c5148f0 commit ac40dbf
Show file tree
Hide file tree
Showing 4 changed files with 130 additions and 23 deletions.
13 changes: 11 additions & 2 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,17 @@ openeo.rest.mlmodel
:inherited-members:


openeo.rest.multiresult
-----------------------


Results
--------

.. automodule:: openeo.rest.result
:members:

.. automodule:: openeo.rest.stac_resource
:members:


.. automodule:: openeo.rest.multiresult
:members: MultiResult
Expand Down
44 changes: 31 additions & 13 deletions openeo/rest/datacube.py
Original file line number Diff line number Diff line change
Expand Up @@ -2369,7 +2369,7 @@ def download(
If outputfile is provided, the result is stored on disk locally, otherwise, a bytes object is returned.
The bytes object can be passed on to a suitable decoder for decoding.
:param outputfile: Optional, an output file if the result needs to be stored on disk.
:param outputfile: Optional, output path to download to.
:param format: Optional, an output format supported by the backend.
:param options: Optional, file format options
:param validate: Optional toggle to enable/prevent validation of the process graphs before execution
Expand Down Expand Up @@ -2515,13 +2515,29 @@ def execute_batch(
**format_options,
) -> BatchJob:
"""
Evaluate the process graph by creating a batch job, and retrieving the results when it is finished.
This method is mostly recommended if the batch job is expected to run in a reasonable amount of time.
Execute the underlying process graph at the backend in batch job mode:
For very long-running jobs, you probably do not want to keep the client running.
- create the job (like :py:meth:`create_job`)
- start the job (like :py:meth:`BatchJob.start() <openeo.rest.job.BatchJob.start>`)
- track the job's progress with an active polling loop
(like :py:meth:`BatchJob.run_synchronous() <openeo.rest.job.BatchJob.run_synchronous>`)
- optionally (if ``outputfile`` is specified) download the job's results
when the job finished successfully
:param outputfile: The path of a file to which a result can be written
.. note::
Because of the active polling loop,
which blocks any further progress of your script or application,
this :py:meth:`execute_batch` method is mainly recommended
for batch jobs that are expected to complete
in a time that is reasonable for your use case.
:param outputfile: Optional, output path to download to.
:param out_format: (optional) File format to use for the job result.
:param title: job title.
:param description: job description.
:param plan: The billing plan to process and charge the job with
:param budget: Maximum budget to be spent on executing the job.
Note that some backends do not honor this limit.
:param additional: additional (top-level) properties to set in the request body
:param job_options: dictionary of job options to pass to the backend
(under top-level property "job_options")
Expand Down Expand Up @@ -2601,17 +2617,19 @@ def create_job(
**format_options,
) -> BatchJob:
"""
Sends the datacube's process graph as a batch job to the back-end
and return a :py:class:`~openeo.rest.job.BatchJob` instance.
Send the underlying process graph to the backend
to create an openEO batch job
and return a corresponding :py:class:`~openeo.rest.job.BatchJob` instance.
Note that the batch job will just be created at the back-end,
it still needs to be started and tracked explicitly.
Use :py:meth:`execute_batch` instead to have the openEO Python client take care of that job management.
Note that this method only *creates* the openEO batch job at the backend,
but it does not *start* it.
Use :py:meth:`execute_batch` instead to let the openEO Python client
take care of the full job life cycle: create, start and track its progress until completion.
:param out_format: output file format.
:param title: job title
:param description: job description
:param plan: The billing plan to process and charge the job with
:param title: job title.
:param description: job description.
:param plan: The billing plan to process and charge the job with.
:param budget: Maximum budget to be spent on executing the job.
Note that some backends do not honor this limit.
:param additional: additional (top-level) properties to set in the request body
Expand Down
9 changes: 7 additions & 2 deletions openeo/rest/result.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@


class SaveResult(StacResource):
"""TODO"""
"""
Alias for :py:class:`~openeo.rest.stac_resource.StacResource`,
returned by methods corresponding to the openEO process ``save_result``, like
:py:meth:`DataCube.save_result() <openeo.rest.datacube.DataCube.save_result>`
and :py:meth:`VectorCube.save_result() <openeo.rest.vectorcube.VectorCube.save_result>`
pass
.. versionadded:: 0.39.0
"""
87 changes: 81 additions & 6 deletions openeo/rest/stac_resource.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,17 @@
class StacResource(_ProcessGraphAbstraction):
"""
Handle for a progress graph node that represents a STAC resource (object with subtype "stac"),
e.g. as returned by `save_result`, or handled by `export_workspace`/`stac_modify`.
e.g. as returned by openEO process ``save_result``,
or handled by openEO processes ``export_workspace``/``stac_modify``.
Refers to a STAC resource of any type (Catalog, Collection, or Item).
It can refer to:
- static STAC resources, e.g. hosted on cloud storage
- dynamic STAC resources made available via a STAC API
- a STAC JSON representation embedded as an argument into an openEO user-defined process
.. versionadded:: 0.39.0
"""

def __init__(self, graph: PGNode, connection: Optional[Connection] = None):
Expand Down Expand Up @@ -59,7 +62,21 @@ def download(
additional: Optional[dict] = None,
job_options: Optional[dict] = None,
):
"""TODO"""
"""
Execute synchronously and download the result (cube).
If outputfile is provided, the result is stored on disk locally, otherwise, a bytes object is returned.
The bytes object can be passed on to a suitable decoder for decoding.
:param outputfile: Optional, output path to download to.
:param validate: Optional toggle to enable/prevent validation of the process graphs before execution
(overruling the connection's ``auto_validate`` setting).
:param additional: additional (top-level) properties to set in the request body
:param job_options: dictionary of job options to pass to the backend
(under top-level property "job_options")
:return: None if the result is stored to disk, or a bytes object returned by the backend.
"""
return self._connection.download(
graph=self.flat_graph(),
outputfile=outputfile,
Expand All @@ -80,7 +97,31 @@ def create_job(
validate: Optional[bool] = None,
log_level: Optional[str] = None,
) -> BatchJob:
"""TODO"""
"""
Send the underlying process graph to the backend
to create an openEO batch job
and return a corresponding :py:class:`~openeo.rest.job.BatchJob` instance.
Note that this method only *creates* the openEO batch job at the backend,
but it does not *start* it.
Use :py:meth:`execute_batch` instead to let the openEO Python client
take care of the full job life cycle: create, start and track its progress until completion.
:param title: job title.
:param description: job description.
:param plan: The billing plan to process and charge the job with.
:param budget: Maximum budget to be spent on executing the job.
Note that some backends do not honor this limit.
:param additional: additional (top-level) properties to set in the request body
:param job_options: dictionary of job options to pass to the backend
(under top-level property "job_options")
:param validate: Optional toggle to enable/prevent validation of the process graphs before execution
(overruling the connection's ``auto_validate`` setting).
:param log_level: Optional minimum severity level for log entries that the back-end should keep track of.
One of "error" (highest severity), "warning", "info", and "debug" (lowest severity).
:return: Handle for the job created at the backend.
"""
return self._connection.create_job(
process_graph=self.flat_graph(),
title=title,
Expand All @@ -107,11 +148,45 @@ def execute_batch(
additional: Optional[dict] = None,
job_options: Optional[dict] = None,
validate: Optional[bool] = None,
auto_add_save_result: bool = True,
show_error_logs: bool = True,
log_level: Optional[str] = None,
) -> BatchJob:
"""TODO"""
"""
Execute the underlying process graph at the backend in batch job mode:
- create the job (like :py:meth:`create_job`)
- start the job (like :py:meth:`BatchJob.start() <openeo.rest.job.BatchJob.start>`)
- track the job's progress with an active polling loop
(like :py:meth:`BatchJob.run_synchronous() <openeo.rest.job.BatchJob.run_synchronous>`)
- optionally (if ``outputfile`` is specified) download the job's results
when the job finished successfully
.. note::
Because of the active polling loop,
which blocks any further progress of your script or application,
this :py:meth:`execute_batch` method is mainly recommended
for batch jobs that are expected to complete
in a time that is reasonable for your use case.
:param outputfile: Optional, output path to download to.
:param title: job title.
:param description: job description.
:param plan: The billing plan to process and charge the job with
:param budget: Maximum budget to be spent on executing the job.
Note that some backends do not honor this limit.
:param additional: additional (top-level) properties to set in the request body
:param job_options: dictionary of job options to pass to the backend
(under top-level property "job_options")
:param validate: Optional toggle to enable/prevent validation of the process graphs before execution
(overruling the connection's ``auto_validate`` setting).
:param log_level: Optional minimum severity level for log entries that the back-end should keep track of.
One of "error" (highest severity), "warning", "info", and "debug" (lowest severity).
:param print: print/logging function to show progress/status
:param max_poll_interval: maximum number of seconds to sleep between job status polls
:param connection_retry_interval: how long to wait when status poll failed due to connection issue
:param show_error_logs: whether to automatically print error logs when the batch job failed.
"""
job = self.create_job(
title=title,
description=description,
Expand Down

0 comments on commit ac40dbf

Please sign in to comment.