Create index templates directly in OpenSearch Dashboards

[spapadop]

Is your feature request related to a problem? Please describe.

Defining index templates is important and currently only possible through APIs.
As a result, validations (e.g. typos) are only applied after the whole index template creation request has been sent. Often times, templates are being pushed using a script, so the errors are possibly fixed too late, possibly requiring reindexing.

Describe the solution you'd like

It would be nice if UI offers a dedicated user-friendly process where people can define their templates. Few ideas on that:

• have a form where users simply fill in the fields (e.g. name, index_pattern) and having them immediately validated, alarming the user for any mistakes (e.g. illegal characters).
• add common schema elements (e.g. geoip) with a simple click from a "common elements" list
• add the possibility of "importing" a test document in order to make sure the template is properly reflected.
• support versioning, in order to have the possibility of tracing back during debugging.

Describe alternatives you've considered

Currently we store, maintain and push index templates through a respective git repo for each cluster. We could perform validations through e.g. GitLab pipelines, but again that's only after a template has been defined. And of course, it can never be as user-friendly as on Dashboards UI.

[wholehope]

I was surprised to know this feature is missing from the Dashboards.

[kamingleung]

Hi @spapadop, thanks for sharing your use cases and suggestions. A few follow-up questions:

• have a form where users simply fill in the fields (e.g. name, index_pattern) and having them immediately validated, alarming the user for any mistakes (e.g. illegal characters).

^ What type of errors do you find see while creating index templates? This will be super helpful on understanding how/where the UI can help users prevent errors or invalid selections beforehand.

• add common schema elements (e.g. geoip) with a simple click from a "common elements" list

^ Would this be related to defining index mapping during template creation?

• add the possibility of "importing" a test document in order to make sure the template is properly reflected.

^ Can you elaborate on this? While a user is creating the template, would this be a dry-run or preview of index creation using the template? What would you expect to see as the result of the dry-run/preview?

[kamingleung]

Hi everyone, I am working on the UX for creating and managing index templates. Looks like there's great interest in having index templates in the Dashboards. I would like to learn more on how you use index templates:

• What are your use cases on using index templates?
• How are you defining your templates?
• What would you like to see to improve your workflow?

Your feedback and suggestions would be super helpful on defining the index template experience. Thanks!

[spapadop]

Hi @kamingleung, apologies for the late answer.

What type of errors do you find see while creating index templates? This will be super helpful on understanding how/where the UI can help users prevent errors or invalid selections beforehand.

• typos when specifying field mapping ("type": "long"). Instead of having to write "type" every time, a nice + button should add that automatically expecting the field name. For the actual value (long, text, keyword) I would be best to have a dropdown list of available types.
• missing some mandatory fields like "index_patterns"
• adding a field with a typo (e.g. "settigns") or a field that has dropped support. All fields of the template should be checked against the acceptable values of the running version. Could also be a dropdown list of some sort.

Would this be related to defining index mapping during template creation?

If I get your question right, yes. There should be a page in UI (I guess under index management plugin) where we should create the templates with the suggested "wizard" (i.e. visual editor, just like when defining index policies)

Can you elaborate on this? While a user is creating the template, would this be a dry-run or preview of index creation using the template? What would you expect to see as the result of the dry-run/preview?

When creating the index template I must be having some kind of documents in mind, that will actually end up on the respective index. Imagine having a doc like:

{
  "field1": "test",
  "field2": "value,
...
  "field10": "value"
}

Once the template is created (or at least when I am happy with the work I've done with it) it would be nice to paste that json (or import it from file). Then the software can analyse the arriving document against the template and produce some errors/warning. For example:

• ERROR You defined 'field3' as integer but it seems to be having floating points
• WARNING You haven't added mapping type for 'field11' that exists in the given document

Hope this helps!

[faph]

+1 for getting this into Dashboards

[devopstales]

+1

[q2dg]

I'd ask for the option to just upload a premade template.json file, "à la curl". Thanks!