From 85333178d853af2234feace2e43f104ad6cc671c Mon Sep 17 00:00:00 2001 From: Zelin Hao Date: Fri, 31 Jan 2025 11:52:47 -0800 Subject: [PATCH 1/3] Add onboarding doc for smoke tests Signed-off-by: Zelin Hao --- ONBOARDING.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/ONBOARDING.md b/ONBOARDING.md index 37cdd4c132..049d5ef444 100644 --- a/ONBOARDING.md +++ b/ONBOARDING.md @@ -35,7 +35,7 @@ Add the new plugin to the [opensearch-plugins meta](https://github.com/opensearc ### Onboard to Test Workflow -1. Update the test configuration file (use 1.3.0 as an example), [opensearch-1.3.0-test.yml](https://github.com/opensearch-project/opensearch-build/blob/opensearch-1.3.0/manifests/1.3.0/opensearch-1.3.0-test.yml), for a particular release, to include your plugin. This test configuration defines full suite of tests - `integ`, `bwc`, that can be run on the plugin. +1. Update the test configuration file (use 1.3.0 as an example), [opensearch-1.3.0-test.yml](https://github.com/opensearch-project/opensearch-build/blob/opensearch-1.3.0/manifests/1.3.0/opensearch-1.3.0-test.yml), for a particular release, to include your plugin. This test configuration defines full suite of tests - `integ`, `bwc`, `smoke` that can be run on the plugin. 2. For integration testing, the `test-workflow` runs integration tests available in the plugin repository. You will need to add `integ-test` config for your plugin in opensearch-1.3.0-test.yml, [example](https://github.com/opensearch-project/opensearch-build/blob/opensearch-1.3.0/manifests/1.3.0/opensearch-1.3.0-test.yml). @@ -47,6 +47,13 @@ Add the new plugin to the [opensearch-plugins meta](https://github.com/opensearc 1. It supports two test configs - `with-security` and `without-security`, which runs test with security plugin enabled and disabled respectively. Choose one or both depending on what your plugin integration tests support. +4. For smoke testing, The test-workflow runs smoke tests defined in the [smoke_tests_spec](src/test_workflow/smoke_test/smoke_tests_spec) directory for each major OpenSearch version. The smoke tests are executed based on the API specification provided in the [OpenSearch API Specification](https://github.com/opensearch-project/opensearch-api-specification) repository. + + 1. The smoke test framework currently supports testing the OpenSearch distribution and its plugins. It ensures that core OpenSearch functionalities and plugin-specific APIs behave as expected before release. + + 2. You will need to add a distinct YAML file for your plugin in the appropriate major version directory under [smoke_tests_spec](src/test_workflow/smoke_test/smoke_tests_spec) and reference the file name in the test manifest for the corresponding component. + + 3. In the YAML file, specify the API path, HTTP method, request body as parameter and heading if `Content-Type` differs from the default `application/json`. Refer to an example test definition [here](src/test_workflow/smoke_test/smoke_tests_spec/2.x/opensearch.yml) for guidance. ## Standalone Component Onboarding From 1db478501fec22c2d400a01ad8117e3caa8e9a1e Mon Sep 17 00:00:00 2001 From: Zelin Hao Date: Thu, 6 Feb 2025 14:08:07 -0800 Subject: [PATCH 2/3] Update onboarding doc Signed-off-by: Zelin Hao --- ONBOARDING.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ONBOARDING.md b/ONBOARDING.md index 049d5ef444..63a06adafa 100644 --- a/ONBOARDING.md +++ b/ONBOARDING.md @@ -47,11 +47,11 @@ Add the new plugin to the [opensearch-plugins meta](https://github.com/opensearc 1. It supports two test configs - `with-security` and `without-security`, which runs test with security plugin enabled and disabled respectively. Choose one or both depending on what your plugin integration tests support. -4. For smoke testing, The test-workflow runs smoke tests defined in the [smoke_tests_spec](src/test_workflow/smoke_test/smoke_tests_spec) directory for each major OpenSearch version. The smoke tests are executed based on the API specification provided in the [OpenSearch API Specification](https://github.com/opensearch-project/opensearch-api-specification) repository. +4. For smoke testing, The test-workflow runs smoke tests defined in the [smoke_tests_spec](src/test_workflow/smoke_test/smoke_tests_spec) directory across all OpenSearch versions and the respective test specification files are organized by major version. The smoke tests are executed based on the API specification provided in the [OpenSearch API Specification](https://github.com/opensearch-project/opensearch-api-specification) repository. - 1. The smoke test framework currently supports testing the OpenSearch distribution and its plugins. It ensures that core OpenSearch functionalities and plugin-specific APIs behave as expected before release. + 1. The smoke test framework currently supports testing the OpenSearch distribution and its plugins. It ensures that core OpenSearch functionalities and plugin-specific APIs behave as expected when bundled into a distribution before release. - 2. You will need to add a distinct YAML file for your plugin in the appropriate major version directory under [smoke_tests_spec](src/test_workflow/smoke_test/smoke_tests_spec) and reference the file name in the test manifest for the corresponding component. + 2. You will need to add a distinct YAML file for your plugin in the appropriate major version directory under [smoke_tests_spec](src/test_workflow/smoke_test/smoke_tests_spec) and reference the file name in the test manifest for the corresponding component. e.g. [security.yml](https://github.com/opensearch-project/opensearch-build/blob/main/src/test_workflow/smoke_test/smoke_tests_spec/default/security.yml) 3. In the YAML file, specify the API path, HTTP method, request body as parameter and heading if `Content-Type` differs from the default `application/json`. Refer to an example test definition [here](src/test_workflow/smoke_test/smoke_tests_spec/2.x/opensearch.yml) for guidance. From 48e4c526b530c2cdefa893d35c85ecd9c073d455 Mon Sep 17 00:00:00 2001 From: Zelin Hao Date: Fri, 7 Feb 2025 14:42:21 -0800 Subject: [PATCH 3/3] Update onboarding Signed-off-by: Zelin Hao --- ONBOARDING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ONBOARDING.md b/ONBOARDING.md index 63a06adafa..8b52fdcd4c 100644 --- a/ONBOARDING.md +++ b/ONBOARDING.md @@ -53,7 +53,7 @@ Add the new plugin to the [opensearch-plugins meta](https://github.com/opensearc 2. You will need to add a distinct YAML file for your plugin in the appropriate major version directory under [smoke_tests_spec](src/test_workflow/smoke_test/smoke_tests_spec) and reference the file name in the test manifest for the corresponding component. e.g. [security.yml](https://github.com/opensearch-project/opensearch-build/blob/main/src/test_workflow/smoke_test/smoke_tests_spec/default/security.yml) - 3. In the YAML file, specify the API path, HTTP method, request body as parameter and heading if `Content-Type` differs from the default `application/json`. Refer to an example test definition [here](src/test_workflow/smoke_test/smoke_tests_spec/2.x/opensearch.yml) for guidance. + 3. In the YAML file, specify the API path, HTTP method, request body as parameter and heading if `Content-Type` differs from the default `application/json`. Refer to an example test definition [here](src/test_workflow/smoke_test/smoke_tests_spec/2.x/opensearch.yml) for guidance. Before adding APIs to the smoke tests, ensure they are already onboarded to the OpenSearch API Specification repository. For onboarding procedures, refer to the [OpenSearch API Specification Developer Guide](https://github.com/opensearch-project/opensearch-api-specification/blob/main/DEVELOPER_GUIDE.md). ## Standalone Component Onboarding