Logstash Ingest Pipelines - Document which processors are not supported

[roaksoax]

Document that the following processors are not supported and why are not supported.

• set_security_user
• inference
• enrich

Ideally, we should be able to document which integrations are supported more-so than processors.

[yaauie]

I agree that what we really should provide is a positive list of which elastic/integrations are supported (although since we do not own them, keeping this in-sync is going to be tricky).

From our existing positive list of what we do support, we could fairly easily script something that validates individual integrations with some caveats:

• we support integrations whose default pipeline is wholly-comprised of supported processors whose if and on_failure clauses are also wholly-comprised of supported processors.
• the pipeline processor is supported if-and-only-if the pipeline it references is fully-supported
  • pipelines referencing {{ IngestPipeline "XYZ" }} in elastic/integrations are effectively package-local, so in PACKAGENAME/data_stream/elasticsearch/ingest_pipeline/default.yml {{IngestPipeline "third-party" }} references PACKAGENAME/data_stream/elasticsearch/ingest_pipeline/third-party.yml:

    IngestManager / EPM replaces '{{IngestPipeline "some-pipeline" }}' with "PACKAGEPREFIX-some-pipeline".

    PACKAGEPREFIX will look like: "logs-PACKAGE.DATASET-VERSION".

    We rename all pipelines before installation by prefixing them with PACKAGEPREFIX so the name of the referenced pipeline matches.
    -- issue comment in elastic/integrations#14

  • if a pipeline otherwise references a pipeline by mustache template we may not be able to recurse
• we do NOT yet support the new reroute processor from Elasticsearch 8.8 (see Logstash Ingest Pipelines - support for "reroute" processor #67)
• we support the remaining processors from IngestCommon
• we support the geoip processor

[lucabelluccini]

The script processors which make use of Processors in ES cannot also be executed...

Caused by: java.lang.IllegalArgumentException: cannot resolve symbol [Processors]

In the Integrations repo we have only 2 occurrences, which could be easily replaced with pure-Painless counterparts