Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pagination fails when datetime is not set #322

Closed
StijnCaerts opened this issue Dec 10, 2024 · 0 comments · Fixed by #323
Closed

Pagination fails when datetime is not set #322

StijnCaerts opened this issue Dec 10, 2024 · 0 comments · Fixed by #323

Comments

@StijnCaerts
Copy link
Collaborator

Describe the bug
Pagination fails when no datetime is set on the STAC Items. This is caused by the way the pagination token for the search_after parameter is constructed.
The pagination token is the string concatenation of all the sort keys. If no datetime value is set on the document, the sort values will look like this from Elasticsearch/Opensearch:

[-9223372036854775808,"<id>","<collection_id>"]

The pagination token (before base64 encoding) is then "-9223372036854775808,<id>,<collection>".

All these sort keys are converted to strings upon decoding the pagination token in the next request:

["-9223372036854775808","<id>","<collection_id>"]

The conversion of the string representation of this number to a number fails in Elasticsearch when the query is executed.

To Reproduce
Steps to reproduce the behavior:

  1. Create a collection and items without properties.datetime value.
  2. Try to search via the /search endpoint.
  3. Request the next page via the pagination links in the response body.
  4. Error message:
{
  "code": "RequestError",
  "description": "RequestError(400, 'search_phase_execution_exception', 'failed to parse date field [-9223372036854775808] with format [strict_date_optional_time||epoch_millis]: [failed to parse date field [-9223372036854775808] with format [strict_date_optional_time||epoch_millis]]')"
}

Expected behavior
Successful retrieval of the next pages.

Additional context
Could be solved by using other format for pagination token, e.g. base64 encoding of JSON string dump instead of string concatenation of values with comma separator.

StijnCaerts added a commit to StijnCaerts/stac-fastapi-elasticsearch that referenced this issue Dec 10, 2024
@StijnCaerts StijnCaerts linked a pull request Dec 10, 2024 that will close this issue
4 tasks
StijnCaerts added a commit that referenced this issue Dec 13, 2024
…323)

**Related Issue(s):**

- #322 

**Description:**
Use base64 encoded JSON string of sort keys as pagination token instead
of comma separated string.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant