Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add backend documentation about configuring ISM notifications #4210

Merged
merged 27 commits into from
Jun 6, 2023

Conversation

vagimeli
Copy link
Contributor

Description

Add new documentation about configuring notifications for ISM API

Issues Resolved

#4170

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Melissa Vagi <[email protected]>
@vagimeli vagimeli self-assigned this May 31, 2023
@vagimeli vagimeli added 2 - In progress Issue/PR: The issue or PR is in progress. v2.8.0 labels May 31, 2023
Signed-off-by: Melissa Vagi <[email protected]>
@vagimeli vagimeli changed the title [DOC] Add ISM API configuring notifications [DOC] Add backend documentation about configuring ISM notifications Jun 2, 2023
vagimeli added 4 commits June 1, 2023 22:30
Signed-off-by: Melissa Vagi <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
@vagimeli vagimeli added 3 - Tech review PR: Tech review in progress and removed 2 - In progress Issue/PR: The issue or PR is in progress. labels Jun 2, 2023
Signed-off-by: Melissa Vagi <[email protected]>
@vagimeli
Copy link
Contributor Author

vagimeli commented Jun 2, 2023

See frontend documentation for reference #4207

vagimeli added 2 commits June 2, 2023 14:03
Signed-off-by: Melissa Vagi <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Copy link
Member

@zhichao-aws zhichao-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, but still need some changes

_im-plugin/ism/api-config-notice.md Outdated Show resolved Hide resolved
_im-plugin/ism/api-config-notice.md Outdated Show resolved Hide resolved
_im-plugin/ism/api-config-notice.md Outdated Show resolved Hide resolved
_im-plugin/ism/api-config-notice.md Outdated Show resolved Hide resolved
_im-plugin/ism/api-config-notice.md Outdated Show resolved Hide resolved
_im-plugin/ism/api-config-notice.md Outdated Show resolved Hide resolved
{
"_id": "LRON:dQlcQ0hQS2mwF-AQ7icCMw:12354",
"lron_config": {
xxxxxxxxx
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a better expression for abbreviations than "xxxx"? e.g. add a line of comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be an appropriate expression example?

Copy link
Member

@zhichao-aws zhichao-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, but still need some changes

Signed-off-by: Melissa Vagi <[email protected]>
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved

Notifications settings allow users to control how they receive notifications about index events. You can use either OpenSearch Dashboards or the REST API to configure notifications. Supported communication channels include Amazon Chime, Amazon Simple Notification Service (Amazon SNS), Amazon Simple Email Service (Amazon SES), email through SMTP, Slack, and custom webhooks. Supported index operations include resize, reindex, open, and force merge. See [Notifications](https://opensearch.org/docs/latest/observing-your-data/notifications/index/) for further details.

To configure configure the notifications settings for long-running index operations using OpenSearch Dashboards, go to **Dev Tools** under the main menu. When creating notifications using the API, the LRON setting can be configured two ways: as a concrete task or as a global and persistent task. The LRON setting with `task_id` is ad hoc and automatically deletes when the task ends. The LRON setting using `action_name` is global and persists and applies to all operations of this action type.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the reference to Dev Tools in Dashboards is confusing. The way to do it using the UI is to go to index management > Notifications. If the user is doing it through the API, they can use curl or dev tools. Normally, we don't direct them to dev tools from an individual API page. How about

Suggested change
To configure configure the notifications settings for long-running index operations using OpenSearch Dashboards, go to **Dev Tools** under the main menu. When creating notifications using the API, the LRON setting can be configured two ways: as a concrete task or as a global and persistent task. The LRON setting with `task_id` is ad hoc and automatically deletes when the task ends. The LRON setting using `action_name` is global and persists and applies to all operations of this action type.
Configuring notifications settings is useful for long-running index operations. When creating long-running operation notifications using the API, you can configure the `lron_config` setting in two ways:
- As a one-time task: If you pass a `task_id` in the `lron_config` object, the task is one-time and the setting automatically deletes when the task ends.
- As a global and persistent task: If you pass an `action_name` in the `lron_config` object, the task is global, persistent, and applies to all operations of this action type.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhichao-aws Some questions:

  • Where do you get the task_id from?
  • In the example below, both task_id and action_name are passed. What happens in this case?
  • For the request and response, we have to provide descriptions for all fields and the possible values for enum fields. Could you help with that? Also, what are the possible values for action_name? For lron_condition, are the possible values only success and failure? Which fields are required and which are optional?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The task id can be obtained by setting the request parameter wait_for_completion as false for open/reindex/resize/force merge. Make some long running operations execute asynchronously and can be tracked by _tasks API OpenSearch#6228. The index operation will return the task id immediately and execute at the background.
  • Infact if task_id has higher priority. If task_id is passed, action_name will be ignored in our code. But it may be useful if user want to search settings about certain actions.
  • For lron_condition, it only has success and failure. Both of them are optional, the value is true by default. lron_condition itself is optional (true, true by default). For action_name, current possible values are "indices:data/write/reindex", "indices:admin/resize", "indices:admin/forcemerge", "indices:admin/open".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config must contain taskID or actionName. And enabled lron_config(i.e. at least there is one true in lron_condition) must contain at least one channel

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, @zhichao-aws! Could you also provide an actual example of a request and response for each section where we have them? Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean create/update/read/delete section, or the cases we talked above?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create/update/read/delete. Basically, wherever we have requests/response placeholders now we should have actual requests/responses.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create/update are actual requests/responses. Here are actual responses of get/delete:
request:
GET _plugins/_im/lron
response:

{
  "lron_configs": [
    {
      "_id": "LRON:indices:admin/open",
      "lron_config": {
        "lron_condition": {
          "success": false,
          "failure": false
        },
        "action_name": "indices:admin/open",
        "channels": []
      }
    },
    {
      "_id": "LRON:indices:data/write/reindex",
      "lron_config": {
        "lron_condition": {
          "success": false,
          "failure": true
        },
        "action_name": "indices:data/write/reindex",
        "channels": [
          {
            "id": "my_chime"
          }
        ]
      }
    }
  ],
  "total_number": 2
}

request:
GET _plugins/_im/lron/LRON:indices:data%2Fwrite%2Freindex
response:

{
  "lron_configs": [
    {
      "_id": "LRON:indices:data/write/reindex",
      "lron_config": {
        "lron_condition": {
          "success": false,
          "failure": true
        },
        "action_name": "indices:data/write/reindex",
        "channels": [
          {
            "id": "my_chime"
          }
        ]
      }
    }
  ],
  "total_number": 1
}

request:
DELETE _plugins/_im/lron/LRON:indices:data%2Fwrite%2Freindex
response(same with delete doc in opensearch normal index):

{
  "_index": ".opensearch-control-center",
  "_id": "LRON:indices:data/write/reindex",
  "_version": 2,
  "result": "deleted",
  "forced_refresh": true,
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "_seq_no": 2,
  "_primary_term": 1
}

_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
Get one document

```bash
GET /_plugins/_im/lron/{lronID}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
GET /_plugins/_im/lron/{lronID}
GET /_plugins/_im/lron/<lronID>

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhichao-aws Is the the _id from the response when you create an lron setting?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a description of the lronID path parameter.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Global: include copy-curl for all requests and make them json instead of bash.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is the _id from the response. For global lron_config, it will be "LRON:"+action_name. For one-time lron_config, it will be "LRON:"+task_id.
Here is one thing I want to call out, there maybe char "/" in action_name, and it may be in _id. If users are using dev tools, they will need to use http encode to transfer "/" to "%2F" in action name. e.g. PUT _plugins/_im/lron/LRON:indices:admin%2Fopen in dev tools

Configuring notifications settings is useful for long-running index operations. When creating long-running operation notifications using the API, you can configure the `lron_config` setting in two ways:

- As a one-time task: If you pass a `task_id` in the `lron_config` object, the task is one-time and the setting automatically deletes when the task ends.
- As a global and persistent task: If you pass an `action_name` in the `lron_config` object, the task is global, persistent, and applies to all operations of this action type.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be precise, If you pass an action_name and not pass the "task_id", then it's global and persistent

@kolchfa-aws
Copy link
Collaborator

@zhichao-aws Could you review the updated documentation and either comment or approve if everything looks good? Thanks!

| `lron_condition` | String | Specifies one or more conditions that must be met for the event to be handled. Conditions are `success` or `failure`. If either condition is set to `true`, you receive a notification through the specified channel when the operation succeeds or fails. If either condition is set to `false`, you do not receive a notification when the operation succeeds or fails. Required. |
| `channels.id` | String | Supported communication channels include Amazon Chime, Amazon Simple Notification Service (Amazon SNS), Amazon Simple Email Service (Amazon SES), email through SMTP, Slack, and custom webhooks. Required. |
| `lron_config._id` | String | <description> |
- **One-time task**: If you pass a `task_id` in the `lron_config` object, the task is one-time and the setting is automatically deleted when the task ends. If you pass both `task_id` and `action_name`, `action_name` is ignored but may be useful to you for searching and debugging notification settings.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the wording "One-time setting" or "Global, persistent setting" may be better. The "task" here may cause confusion with the "task_id" task.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhichao-aws Implemented, thanks. The PR is ready for your final review.

Signed-off-by: Fanit Kolchina <[email protected]>
Copy link
Member

@zhichao-aws zhichao-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, looks good to me.

@kolchfa-aws kolchfa-aws added 4 - Doc review PR: Doc review in progress and removed 3 - Tech review PR: Tech review in progress labels Jun 6, 2023
Copy link
Contributor Author

@vagimeli vagimeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kolchfa-aws minimal edits

{
"_id": "LRON:dQlcQ0hQS2mwF-AQ7icCMw:12354",
"lron_config": {
xxxxxxxxx
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be an appropriate expression example?

_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
@vagimeli vagimeli assigned vagimeli and kolchfa-aws and unassigned vagimeli Jun 6, 2023
Copy link
Collaborator

@kolchfa-aws kolchfa-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

kolchfa-aws and others added 2 commits June 6, 2023 08:41
Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
@kolchfa-aws kolchfa-aws added 6 - Done but waiting to merge PR: The work is done and ready to merge and removed 4 - Doc review PR: Doc review in progress labels Jun 6, 2023
Copy link
Contributor

@hdhalter hdhalter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few suggestions. Looks good!

_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved

## Configuring notification settings

When creating long-running operation notifications using the API, you can configure the `lron_config` using the `task_id` and `action_name` parameters as follows:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When creating long-running operation notifications using the API, you can configure the `lron_config` using the `task_id` and `action_name` parameters as follows:
When creating long-running operation notifications using the API, you can configure the `lron_config` object using the `task_id` and `action_name` parameters as follows:

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded.

_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
_im-plugin/notifications-settings.md Outdated Show resolved Hide resolved
kolchfa-aws and others added 6 commits June 6, 2023 11:25
Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
@kolchfa-aws kolchfa-aws merged commit 78297a1 into main Jun 6, 2023
@kolchfa-aws kolchfa-aws changed the title [DOC] Add backend documentation about configuring ISM notifications Add backend documentation about configuring ISM notifications Jun 6, 2023
harshavamsi pushed a commit to harshavamsi/documentation-website that referenced this pull request Oct 31, 2023
…pensearch-project#4210)

* Add new documentation page

Signed-off-by: Melissa Vagi <[email protected]>

* Add new documentation

Signed-off-by: Melissa Vagi <[email protected]>

* Writing

Signed-off-by: Melissa Vagi <[email protected]>

* Writing

Signed-off-by: Melissa Vagi <[email protected]>

* Fix code example format

Signed-off-by: Melissa Vagi <[email protected]>

* Copy editing

Signed-off-by: Melissa Vagi <[email protected]>

* Copy editing

Signed-off-by: Melissa Vagi <[email protected]>

* Spelling error

Signed-off-by: Melissa Vagi <[email protected]>

* Add console sentence

Signed-off-by: Melissa Vagi <[email protected]>

* Address tech review feedback

Signed-off-by: Melissa Vagi <[email protected]>

* Address doc review comments

Signed-off-by: Melissa Vagi <[email protected]>

* Address doc review changes

Signed-off-by: Melissa Vagi <[email protected]>

* Address doc review comments

Signed-off-by: Melissa Vagi <[email protected]>

* Address doc review feedback

Signed-off-by: Melissa Vagi <[email protected]>

* Address doc review feedback

Signed-off-by: Melissa Vagi <[email protected]>

* Add more info to notification settings

Signed-off-by: Fanit Kolchina <[email protected]>

* Reword for clarity

Signed-off-by: Fanit Kolchina <[email protected]>

* Implemented tech review comments

Signed-off-by: Fanit Kolchina <[email protected]>

* Update _im-plugin/notifications-settings.md

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

* Update _im-plugin/notifications-settings.md

Co-authored-by: Melissa Vagi <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

* Update _im-plugin/notifications-settings.md

Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

* Update _im-plugin/notifications-settings.md

Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

* Update _im-plugin/notifications-settings.md

Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

* Update _im-plugin/notifications-settings.md

Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

* Update _im-plugin/notifications-settings.md

Co-authored-by: Heather Halter <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

* Doc review comments

Signed-off-by: Fanit Kolchina <[email protected]>

---------

Signed-off-by: Melissa Vagi <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Fanit Kolchina <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
Co-authored-by: Heather Halter <[email protected]>
@Naarcha-AWS Naarcha-AWS deleted the api-config-notice branch March 28, 2024 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6 - Done but waiting to merge PR: The work is done and ready to merge release-notes PR: Include this PR in the automated release notes v2.8.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC] Add documentation for api of configuring notifications on Index Management operations
4 participants