From 242632095162f93c7388c83018eb5d52117aa937 Mon Sep 17 00:00:00 2001 From: Romain Bioteau Date: Wed, 31 Jul 2024 14:42:00 +0200 Subject: [PATCH 1/9] doc(purgeTool): changelog and other --- .../runtime/pages/purge-tool-changelog.adoc | 39 +++- modules/runtime/pages/purge-tool.adoc | 194 +++++++++++++++--- 2 files changed, 205 insertions(+), 28 deletions(-) diff --git a/modules/runtime/pages/purge-tool-changelog.adoc b/modules/runtime/pages/purge-tool-changelog.adoc index 798a353249..01cb1e3537 100644 --- a/modules/runtime/pages/purge-tool-changelog.adoc +++ b/modules/runtime/pages/purge-tool-changelog.adoc @@ -6,6 +6,40 @@ This is the changelog of the purge tool. The purge tool is used to remove data from bonita archive tables. It is useful for big production environments. +== 2.0.0 - August ?, 2024 + +Breaking changes: + +- Java 17+ is now required to run the tool. +- Properties in the `application.properties` file have been renamed. Please update your configuration file accordingly. + +New features: + +- Improve the CLI user experience. +- Add a new `list` command to list the archived process instances that can be deleted. +- Add a new `delete` command to delete archived process instances with more options for better flexibility. +- The `delete` command now supports 2 deletion modes: `batch-delete` and `copy-truncate`. +- Delete operation is now executed in batch for a better control of the deletion process. +- Add a new option to the `delete` command to set the batch size. +- Add a new option to the `delete` command to set a timeout interval between each batch. +- Add a new option to the `delete` command to use a period (e.g.: 1d, 2w, 3m, 4y) instead of an exact date timestamp to filter instances. +- Process definition id is now optional in the `delete` command. +- Add a new `delete-file-input` command to delete archived file input values. + +Fixes: + +- The CLI can be run from `bin/` or root folder. +- Fix the date format display + +Others: + +- deps: update Springboot version to `3.3.2` +- deps: update MySQL connector to `8.4.0` +- deps: update PostgreSQL driver to `42.7.3` +- deps: update Oracle driver to `21.14.0.0` +- deps: update MS SQL Server driver to `9.4.1.jre11` +- deps: update Exposed JDBC to `0.52.0` + == 1.0.9 - June 11, 2024 @@ -19,9 +53,10 @@ Fixes: Now the tool **MUST** be run from the **root folder** by calling: -```shell +[source,shell] +-- bin/bonita-purge-tool[.sh/.bat] -``` +-- It cannot be run from the **bin/** folder anymore. Fixes: diff --git a/modules/runtime/pages/purge-tool.adoc b/modules/runtime/pages/purge-tool.adoc index b0117ae806..a33003d6c5 100644 --- a/modules/runtime/pages/purge-tool.adoc +++ b/modules/runtime/pages/purge-tool.adoc @@ -2,65 +2,207 @@ :page-aliases: ROOT:purge-tool.adoc :description: Bonita Purge Tool provides the capability to purge finished (archived) process instances from Bonita Runtime environment. +[NOTE] +==== +For Subscription editions only. + +This tool is not available publicly. Use {cscDownloadUrl}[Bonita Customer Service Center] to download it. +==== + + Bonita Purge Tool provides the capability to purge finished (archived) process instances from Bonita Runtime environment. By default, all archives are preserved forever in Bonita runtime, but if your functional context allows you to remove old unused process instances (for example, if you only need to keep a history of last 6 months), use this tool to clean up your Bonita database. - [WARNING] ==== The purge tool doesn't delete documents (stored in the DOCUMENT table) from the platform. It will only remove the mapping between the archived cases and the document itself. If you need to reduce the size of the Document table in the engine database, please refer to the documentation: xref:ROOT:handling-documents.adoc#delete_document_archived_case[Delete documents of archived cases based on archive date] ==== -[NOTE] -==== -For Subscription editions only. + -This tool is not available publicly. Use {cscDownloadUrl}[Bonita Customer Service Center] to download it. -==== - == Pre-requisites -This tool can be run on a Bonita runtime environment in a version greater than or equal to 7.7.0. + -Bonita runtime environment should be shut down when running this tool, i.e. Bonita server should be stopped. +This tool requires a Java 17+ runtime environment to run. + +This tool can be run on a Bonita runtime environment in a version greater than or equal to 7.7.0. + +[CAUTION] +==== +When deleting archives using the `copy-truncate` mode, the Bonita runtime connected to the database must be shut down when running this tool. +==== == Configuration -Once {cscDownloadUrl}[downloaded Bonita Purge Tool], unzip it somewhere and go into the main directory. + +Once {cscDownloadUrl}[downloaded], unzip it somewhere and go into the main directory. + Enter your database configuration properties in the file `application.properties` +.Configuration example +[source,properties] +---- +# Database configuration +database.vendor=postgres +database.name=bonita +database.username=db_user +database.password=secret +database.host=localhost +database.port=5432 +---- + +[NOTE] +==== +It is also possible to use environment variables to set these properties. + +.Environment variables example +[source,shell] +---- +DATABASE_PASSWORD=secret ./bin/bonita-purge-tool list --older-than 6m +---- +==== + == Run Bonita Purge Tool -This command will delete all archived process instances belonging to the process identified by *PROCESS_DEFINITION_ID*, -that are finished since at least *OLDEST_DATE_TIMESTAMP*. +The Bonita Purge Tool is a command-line tool. + +To run it, open a terminal and go to the directory where you unzipped the tool. + +View the usage information by running the following command: + +[tabs] +==== +Unix:: ++ +[source,shell] +---- +./bin/bonita-purge-tool --help +---- + +Windows:: ++ +[source,shell] +---- +./bin/bonita-purge-tool.bat --help +---- +==== + +=== `list` command + +This command lists all existing process definitions that have root process instances archived for given date filter. + +.List process definitions with archived root process instances older than 6 months +[tabs] +==== +Unix:: ++ +[source,shell] +---- +./bin/bonita-purge-tool list --older-than 6m +---- + +Windows:: ++ +[source,shell] +---- +./bin/bonita-purge-tool.bat list --older-than 6m +---- +==== + +=== `delete` command -example (Unix): +This command deletes archived process instances and their related archived elements (flownodes, data, comments, etc.) for given date filter. +.Delete all archived process instances older than 6 months +[tabs] +==== +Unix:: ++ [source,shell] ---- -# If no PROCESS_DEFINITION_ID parameter is given, the program lists all existing process definitions and stops: -bin/bonita-purge-tool +./bin/bonita-purge-tool delete --older-than 6m +---- -bin/bonita-purge-tool [] +Windows:: ++ +[source,shell] +---- +./bin/bonita-purge-tool.bat delete --older-than 6m ---- +==== -example (Windows): +.Delete archived process instances older than 6 months for a specific process definition +[tabs] +==== +Unix:: ++ +[source,shell] +---- +./bin/bonita-purge-tool delete --older-than 6m --process-definition-id 1234567890 +---- +Windows:: ++ [source,shell] ---- -# If no PROCESS_DEFINITION_ID parameter is given, the program lists all existing process definitions and stops: -bin/bonita-purge-tool.bat +./bin/bonita-purge-tool.bat delete --older-than 6m --process-definition-id 1234567890 +---- +==== -bin/bonita-purge-tool.bat []` +.Delete archived process instances before 5 July 2022 +[tabs] +==== +Unix:: ++ +[source,shell] +---- +./bin/bonita-purge-tool delete --before-date 1656979200000 ---- -An optional *TENANT_ID* parameter can be given if the platform uses multiple tenants to specify on which tenant should the process instances be deleted. -If multi-tenancy is used and if the TENANT_ID is not set, an error is issued, and the program stops. +Windows:: ++ +[source,shell] +---- +./bin/bonita-purge-tool.bat delete --before-date 1656979200000 +---- +==== +NOTE: The `--before-date` parameter must be in https://www.epochconverter.com/[milliseconds since the epoch]. + +==== Delete modes + +===== `batch-delete` + +By default, the tool uses the `batch-delete` mode to delete rows in database tables. + +This mode will be slower than the copy-truncate mode, but it doesn't require the Runtime to be shutdown and can be stopped and resumed at any time. + +In this mode, each deleted batch is committed in database. + +The batch size and the timeout interval between each batch can be configured using the `--batch-size` and `--timeout-interval` options on the `delete` command. + +Fine tune the batch size and timeout interval depending on your database configuration and the volume of data to delete. By default, the batch size is `5000` and the timeout interval is `0` ms. + +===== `copy-truncate` + +In this mode, the tool will copy the rows to keep in a temporary table, then truncate and drop the original table and rename the temporary table and recreate all required constraints and indices. + +Due to the efficiency of the TRUNCATE command, this method should be faster in most cases, especially when the number of rows to delete is higher than the row to keep in the table, but it requires the Runtime to be stopped during the operation. + +Use the `--delete-mode copy-truncate` option to use this mode with the `delete` command. -*OLDEST_DATE_TIMESTAMP* must be a Timestamp (in milliseconds) from which all process instances that finished before that date will be deleted. + -Unfinished process instances and process instances that finished after that date will not be affected. + -Its format is a standard Java timestamp since https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/time/Instant.html#EPOCH[EPOCH] (in milliseconds). -You can use websites such as https://www.epochconverter.com/[Epoch Converter] to format such a timestamp. +=== `delete-file-input` command + +Delete all archived contract file input values. + +.Delete archived process instances before 5 July 2022 +[tabs] +==== +Unix:: ++ +[source,shell] +---- +./bin/bonita-purge-tool delete-file-input +---- + +Windows:: ++ +[source,shell] +---- +./bin/bonita-purge-tool.bat delete-file-input +---- +==== == Deletion strategy From 2247265aab29e62b048460b7a9733b5ae222dca8 Mon Sep 17 00:00:00 2001 From: Romain Bioteau Date: Mon, 26 Aug 2024 11:23:33 +0200 Subject: [PATCH 2/9] Apply suggestions from code review --- modules/runtime/pages/purge-tool-changelog.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/runtime/pages/purge-tool-changelog.adoc b/modules/runtime/pages/purge-tool-changelog.adoc index 01cb1e3537..c321633017 100644 --- a/modules/runtime/pages/purge-tool-changelog.adoc +++ b/modules/runtime/pages/purge-tool-changelog.adoc @@ -33,12 +33,12 @@ Fixes: Others: -- deps: update Springboot version to `3.3.2` +- deps: update Springboot version to `3.3.3` - deps: update MySQL connector to `8.4.0` -- deps: update PostgreSQL driver to `42.7.3` -- deps: update Oracle driver to `21.14.0.0` +- deps: update PostgreSQL driver to `42.7.4` +- deps: update Oracle driver to `21.15.0.0` - deps: update MS SQL Server driver to `9.4.1.jre11` -- deps: update Exposed JDBC to `0.52.0` +- deps: update Exposed JDBC to `0.53.0` == 1.0.9 - June 11, 2024 From 5c6ae7c89cc6b979415e37a852453ee503fd1c99 Mon Sep 17 00:00:00 2001 From: Romain Bioteau Date: Mon, 26 Aug 2024 11:43:46 +0200 Subject: [PATCH 3/9] Update modules/runtime/pages/purge-tool.adoc Co-authored-by: emmanuel.duchastenier@bonitasoft.com --- modules/runtime/pages/purge-tool.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/runtime/pages/purge-tool.adoc b/modules/runtime/pages/purge-tool.adoc index a33003d6c5..17c7077766 100644 --- a/modules/runtime/pages/purge-tool.adoc +++ b/modules/runtime/pages/purge-tool.adoc @@ -178,7 +178,7 @@ Fine tune the batch size and timeout interval depending on your database configu In this mode, the tool will copy the rows to keep in a temporary table, then truncate and drop the original table and rename the temporary table and recreate all required constraints and indices. -Due to the efficiency of the TRUNCATE command, this method should be faster in most cases, especially when the number of rows to delete is higher than the row to keep in the table, but it requires the Runtime to be stopped during the operation. +Due to the efficiency of the TRUNCATE command, this method should be faster in most cases, especially when the number of rows to delete is higher than the number of rows to keep in the table, but it requires the Runtime to be stopped during the operation. Use the `--delete-mode copy-truncate` option to use this mode with the `delete` command. From ac7fee113cacea6d79ef58e463df0550af7ece4d Mon Sep 17 00:00:00 2001 From: Romain Bioteau Date: Mon, 26 Aug 2024 11:43:58 +0200 Subject: [PATCH 4/9] Update modules/runtime/pages/purge-tool.adoc Co-authored-by: emmanuel.duchastenier@bonitasoft.com --- modules/runtime/pages/purge-tool.adoc | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/modules/runtime/pages/purge-tool.adoc b/modules/runtime/pages/purge-tool.adoc index 17c7077766..87827d5c76 100644 --- a/modules/runtime/pages/purge-tool.adoc +++ b/modules/runtime/pages/purge-tool.adoc @@ -184,7 +184,15 @@ Use the `--delete-mode copy-truncate` option to use this mode with the `delete` === `delete-file-input` command -Delete all archived contract file input values. +Delete all archived contract file input values. + +In other words, delete all rows in table `arch_contract_data` corresponding to contract data of type `File` (in Studio) or `org.bonitasoft.engine.bpm.contract.FileInputValue` in Bonita Engine. + +These data are not used by Bonita and can take a large amout of space in your database, so deleting them is advised. + +[NOTE] +==== +From Bonita 10.4, these data are no more archived, so it will not be necessary anymore to run this command. +==== + .Delete archived process instances before 5 July 2022 [tabs] From 07ae696c66243eafcd733020d4236e13a62c91f2 Mon Sep 17 00:00:00 2001 From: Romain Bioteau Date: Mon, 26 Aug 2024 11:44:19 +0200 Subject: [PATCH 5/9] Update modules/runtime/pages/purge-tool.adoc Co-authored-by: emmanuel.duchastenier@bonitasoft.com --- modules/runtime/pages/purge-tool.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/runtime/pages/purge-tool.adoc b/modules/runtime/pages/purge-tool.adoc index 87827d5c76..1856e0878b 100644 --- a/modules/runtime/pages/purge-tool.adoc +++ b/modules/runtime/pages/purge-tool.adoc @@ -194,7 +194,7 @@ From Bonita 10.4, these data are no more archived, so it will not be necessary a ==== -.Delete archived process instances before 5 July 2022 +.Delete all archived contract file input values [tabs] ==== Unix:: From 01b4023599b66624f5e57dc75247df7de3b66c88 Mon Sep 17 00:00:00 2001 From: Romain Bioteau Date: Wed, 11 Sep 2024 08:30:46 +0200 Subject: [PATCH 6/9] Apply suggestions from code review --- modules/runtime/pages/purge-tool-changelog.adoc | 2 +- modules/runtime/pages/purge-tool.adoc | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/runtime/pages/purge-tool-changelog.adoc b/modules/runtime/pages/purge-tool-changelog.adoc index c321633017..2320d941d9 100644 --- a/modules/runtime/pages/purge-tool-changelog.adoc +++ b/modules/runtime/pages/purge-tool-changelog.adoc @@ -24,7 +24,7 @@ New features: - Add a new option to the `delete` command to set a timeout interval between each batch. - Add a new option to the `delete` command to use a period (e.g.: 1d, 2w, 3m, 4y) instead of an exact date timestamp to filter instances. - Process definition id is now optional in the `delete` command. -- Add a new `delete-file-input` command to delete archived file input values. +- Add a new `delete-fileinput-content` command to delete archived file input values. Fixes: diff --git a/modules/runtime/pages/purge-tool.adoc b/modules/runtime/pages/purge-tool.adoc index 1856e0878b..02fafe3aaa 100644 --- a/modules/runtime/pages/purge-tool.adoc +++ b/modules/runtime/pages/purge-tool.adoc @@ -194,21 +194,21 @@ From Bonita 10.4, these data are no more archived, so it will not be necessary a ==== -.Delete all archived contract file input values +.Delete all archived contract file input content [tabs] ==== Unix:: + [source,shell] ---- -./bin/bonita-purge-tool delete-file-input +./bin/bonita-purge-tool delete-fileinput-content ---- Windows:: + [source,shell] ---- -./bin/bonita-purge-tool.bat delete-file-input +./bin/bonita-purge-tool.bat delete-fileinput-content ---- ==== From de2f7fbf48a9b053a0aa0d7f75ac93c001915efd Mon Sep 17 00:00:00 2001 From: Romain Bioteau Date: Wed, 11 Sep 2024 08:34:34 +0200 Subject: [PATCH 7/9] Apply suggestions from code review --- modules/runtime/pages/purge-tool-changelog.adoc | 4 ++-- modules/runtime/pages/purge-tool.adoc | 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/modules/runtime/pages/purge-tool-changelog.adoc b/modules/runtime/pages/purge-tool-changelog.adoc index 2320d941d9..518eb7b42e 100644 --- a/modules/runtime/pages/purge-tool-changelog.adoc +++ b/modules/runtime/pages/purge-tool-changelog.adoc @@ -24,7 +24,7 @@ New features: - Add a new option to the `delete` command to set a timeout interval between each batch. - Add a new option to the `delete` command to use a period (e.g.: 1d, 2w, 3m, 4y) instead of an exact date timestamp to filter instances. - Process definition id is now optional in the `delete` command. -- Add a new `delete-fileinput-content` command to delete archived file input values. +- Add a new `delete-fileinput-content` command to delete archived contract file input content (Not supported for SQL Server). Fixes: @@ -38,7 +38,7 @@ Others: - deps: update PostgreSQL driver to `42.7.4` - deps: update Oracle driver to `21.15.0.0` - deps: update MS SQL Server driver to `9.4.1.jre11` -- deps: update Exposed JDBC to `0.53.0` +- deps: update Exposed JDBC to `0.54.0` == 1.0.9 - June 11, 2024 diff --git a/modules/runtime/pages/purge-tool.adoc b/modules/runtime/pages/purge-tool.adoc index 02fafe3aaa..8f229c8d2f 100644 --- a/modules/runtime/pages/purge-tool.adoc +++ b/modules/runtime/pages/purge-tool.adoc @@ -212,6 +212,8 @@ Windows:: ---- ==== +WARNING: `delete-fileinput-content` command is not supported for SQLServer database. + == Deletion strategy You need to have in mind 2 precepts to understand how this tool works: From f17c985abc046a81ed5e1de3c45ed5e86ef293a8 Mon Sep 17 00:00:00 2001 From: Romain Bioteau Date: Wed, 11 Sep 2024 08:35:02 +0200 Subject: [PATCH 8/9] Apply suggestions from code review --- modules/runtime/pages/purge-tool-changelog.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/runtime/pages/purge-tool-changelog.adoc b/modules/runtime/pages/purge-tool-changelog.adoc index 518eb7b42e..b23722fe01 100644 --- a/modules/runtime/pages/purge-tool-changelog.adoc +++ b/modules/runtime/pages/purge-tool-changelog.adoc @@ -6,7 +6,7 @@ This is the changelog of the purge tool. The purge tool is used to remove data from bonita archive tables. It is useful for big production environments. -== 2.0.0 - August ?, 2024 +== 2.0.0 - September 11, 2024 Breaking changes: From 1ef9cc8f16df81355999d9e6bf809f849af6adb9 Mon Sep 17 00:00:00 2001 From: Romain Bioteau Date: Thu, 24 Oct 2024 15:30:48 +0200 Subject: [PATCH 9/9] update release date --- modules/runtime/pages/purge-tool-changelog.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/runtime/pages/purge-tool-changelog.adoc b/modules/runtime/pages/purge-tool-changelog.adoc index b23722fe01..a3789ea6a8 100644 --- a/modules/runtime/pages/purge-tool-changelog.adoc +++ b/modules/runtime/pages/purge-tool-changelog.adoc @@ -6,7 +6,7 @@ This is the changelog of the purge tool. The purge tool is used to remove data from bonita archive tables. It is useful for big production environments. -== 2.0.0 - September 11, 2024 +== 2.0.0 - October 24, 2024 Breaking changes: @@ -33,12 +33,12 @@ Fixes: Others: -- deps: update Springboot version to `3.3.3` +- deps: update Springboot version to `3.3.4` - deps: update MySQL connector to `8.4.0` - deps: update PostgreSQL driver to `42.7.4` - deps: update Oracle driver to `21.15.0.0` - deps: update MS SQL Server driver to `9.4.1.jre11` -- deps: update Exposed JDBC to `0.54.0` +- deps: update Exposed JDBC to `0.55.0` == 1.0.9 - June 11, 2024