encapsulate {httr} stuff in helper functions, test against latest Elasticsearch

[jameslamb]

Contributes to #229

• consolidates all {httr} stuff into internal helperfuns
• adds support for the latest Elasticsearch version (8.17.12)
  • ref: https://hub.docker.com/_/elasticsearch/tags

Notes for Reviewers

Proposing adding the latest Elasticsearch version to be sure we're covering that before I switch out the do-HTTP-stuff package, so we know that if we see testing failures it's a result of that refactoring.

How I tested this

I ran the instructions in CONTRIBUTING.md, on my M2 macbook (arm64).

uptasticsearch/CONTRIBUTING.md

Lines 121 to 133 in 165ca03

	```shell
	# Start up Elasticsearch on localhost:9200 and seed it with data
	./setup_local.sh 7.12.1
	# Run tests
	make test
	# Get test coverage and generate coverage report
	make coverage
	# Tear down the container and remove testing files
	./cleanup_local.sh
	```

[jameslamb]

had a problem deploying to github-pages 2 hours ago — with GitHub Actions Failure

This can safely be ignored. Fixed with #246

[jameslamb]

Putting this somewhere on the diff to thread it. Reposting what I'd written 2 days ago:

hmmmmm, the integration tests are failing only for Elasticsearch 7.17.22... not any of the versions before or after.

  ── 3. Error ('test-integration.R:338:9'): .get_aliases and get_fields work as ex
  <http_400/http_error/error/condition>
  Error in `.stop_for_status(result)`: Bad Request (HTTP 400).
  Backtrace:

(build link)

Haven't been able to reproduce that locally.

[jameslamb]

Finally found it!

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Indices [.geoip_databases] use and access is reserved for system operations"}],"type":"illegal_argument_exception","reason":"Indices [.geoip_databases] use and access is reserved for system operations"},"status":400}-

(build link)

It looks like for a call like get_fields(..., es_indices = "_all"), this .geoip_databases index is showing up in the list of indices, right here:

uptasticsearch/r-pkg/R/get_fields.R

Lines 51 to 65 in 52db003

	# The use of "_all" to indicate "all indices" was removed in Elasticsearch 7.
	if (as.integer(major_version) > 6 && indices == "_all") {
	log_warn(sprintf(
	paste0(
	"You are running Elasticsearch version '%s.x'. _all is not supported in this version."
	, " Pulling all indices with 'POST /_cat/indices' for you." # nolint[non_portable_path]
	)
	, major_version
	))
	res <- .request(
	verb = "GET"
	, url = sprintf("%s/_cat/indices?format=json", es_url)
	, config = list()
	, body = NULL
	)

And then it's failing right here, with the error seen above

uptasticsearch/r-pkg/R/get_fields.R

Lines 82 to 89 in 52db003

	log_info(paste("Getting indexed fields for indices:", indices))
	result <- .request(
	verb = "GET"
	, url = es_url
	, config = httr::add_headers(c("Content-Type" = "application/json")) # nolint[non_portable_path]
	, body = NULL
	)

[jameslamb]

Someone reported this to elasticsearch, and the outcome of it was ".geoip_databases was supposed to be in GET /_cat/indices all long": elastic/elasticsearch#74687

And yet, it can't be queries with /_search and other APIS... so for example, Kibana started skipping it: elastic/kibana#104155

The full list (being updated as Elasticsearch changes) is here: elastic/elasticsearch#50251

[jameslamb]

Ran tests on a couple versions (https://github.com/uptake/uptasticsearch/actions/runs/13405276725/job/37443973224?pr=245) ... it seems to me like 0 "system indices" are returned by GET /_cat/indices in v7.01, v8.01, or v8.17.2.

But who knows, maybe that is an artifact of the default settings in the different container images, not actual changes in Elasticsearch.

And maybe we're only seeing .geoip_databases and not other system indices because in the 7.17.22 images, the ES functionality that uses whatever that is is enabled.

[jameslamb]

One more: https://discuss.elastic.co/t/why-cat-indices-shows-system-indices/337837

The index setting index.hidden will determine if the index is displayed or hidden by default. Default is false.

[jameslamb]

Another long-lived bug! This has been a GET request since the moment it was introduced back in 2019 😬

ref: #161