Adding support for opensearch

docker-compose.cicd.yml

@@ -9,7 +9,7 @@ services:
       - "3000:3000"
     depends_on:
       - fhir
-      - es
+      - opensearch
     restart: always
     environment:
       - NODE_ENV=cicd
@@ -24,22 +24,29 @@ services:
     # networks:
     #   - opencr
 
-  es:
-    container_name: es
-    image: intrahealth/elasticsearch:latest
+  opensearch:
+    container_name: opensearch
+    image: intrahealth/opensearch:latest
     environment:
-      - node.name=es01
+      - cluster.name=opensearch-cluster
+      - node.name=opensearch-node1
       - discovery.type=single-node
       - bootstrap.memory_lock=true
-      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
+      - plugins.security.disabled=true
+      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
     ulimits:
       memlock:
         soft: -1
         hard: -1
+      nofile:
+        soft: 65536
+        hard: 65536
     volumes:
-      - data01:/usr/share/elasticsearch/data
+      - data01:/usr/share/opensearch/data
+
     ports:
       - 9200:9200
+      - 9600:9600
     # networks:
     #   - opencr
 

docker/opensearch/Dockerfile

@@ -0,0 +1,5 @@
+FROM opensearchproject/opensearch:2.1.0
+
+RUN /usr/share/opensearch/bin/opensearch-plugin install analysis-phonetic
+
+RUN /usr/share/opensearch/bin/opensearch-plugin install --batch https://github.com/DigitalSQR/record-linkage/releases/download/v2.1.0.0/record-linkage.zip

docker/opensearch/dockerhub.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -ex
+
+# automate tagging with the short commit hash
+docker build --no-cache -t intrahealth/opensearch:$(git rev-parse --short HEAD) .
+docker tag intrahealth/opensearch:$(git rev-parse --short HEAD) intrahealth/opensearch
+docker push intrahealth/opensearch:$(git rev-parse --short HEAD)
+docker push intrahealth/opensearch:latest

docs/admin/architecture.md

@@ -9,9 +9,8 @@ The OpenCR architecture includes:
 
 * The [**OpenCR Service**](https://github.com/intrahealth/client-registry): The API for managing queries, routing traffic to the components, and overall entrypoint. It is written in Node JS.
 * The [**HAPI FHIR Server**](http://hapifhir.io): HAPI is the reference FHIR server in Java and scalable into production environments.
-* The [**ElasticSearch**](http://elastic.co/products/elasticsearch): Elasticsearch is a powerful search engine that is highly performant.
+* Either [**OpenSearch**](https://opensearch.org/) or [**ElasticSearch**](http://elastic.co/products/elasticsearch): Both OpenSearch and Elasticsearch are powerful search engines that are highly performant. One of them must be installed, not both, however we do recommend OpenSearch to be used only because of the license restrictions with ElasticSearch.
 * An **optional UI** to view and break matches between records, and view matching histories (audit events).
 * The [**Open Health Information Mediator (OpenHIM)**](http://openhim.org) (Optional): The OpenHIM is the entrypoint for POS systems, and includes authentication (are you who you say you are?), authorization (what roles do you have permission to fulfill?), and auditing of all transactions. OpenHIM is optional but the administrator must manage users and node access in some manner if not with OpenHIM.
 
 ![Alt text](../images/production_architecture.png "Production Architecture")
-

docs/admin/configuration.md

@@ -5,11 +5,14 @@ Often there are many records of the same person but in many people in different
 !!! caution
     The Client Registry does not store clinical information. Having the Client Registry enables the ability to create a Shared Health Record in the future.
 
+!!! note
+    OpenSearch and ElasticSearch uses exactly the same technology and have almost the same configuration, i.e ports etc. The configuration file of OpenCR uses parameter elasticsearch to refer to either ElasticSearch or OpenSearch.
+
 The Client Registry stores the patient demographic data submitted to it in queries. The Client Registry stores demographic data at least in the HAPI FHIR Server, which can have any database backend an implementer chooses to use.
 
-ElasticSearch (ES) is an optional search engine, and requires configuration. ES can also store patient data fields selectably.
+OpenSearch/ElasticSearch (ES) is a required search engine for matching, and may requires configuration.
 
-JSON files are used to configure the system. Later iterations will support environment variables and a graphical interface. See https://github.com/openhie/client-registry/tree/master/server/config for example configuration files discussed here.
+JSON files are used to configure the system. Later iterations will support environment variables and a graphical interface. See <https://github.com/openhie/client-registry/tree/master/server/config> for example configuration files discussed here.
 
 ## Deciding Between a Standalone or Mediator Configuration
 
@@ -42,7 +45,7 @@ In standalone mode, the server runs TLS by default, and requires signed certific
 The default ports are as follows:
 
 - **3000**: Client Registry Service
-- **9200**: ElasticSearch (closed to external)
+- **9200**: OpenSearch/ElasticSearch (closed to external)
 - **8080**: HAPI FHIR Server (closed to external)
 
 In `server/config/config_development_template.json` there is a template for configuration.
@@ -128,7 +131,7 @@ Contents of `server/config/config_development_template.json`
 
 The currently supported FHIR version is R4.
 
-`fhirServer.baseURL` is the default. Note that it may change depending on the way HAPI is installed. It may, for example, default to a baseURL of http://localhost:8080/baseR4/.
+`fhirServer.baseURL` is the default. Note that it may change depending on the way HAPI is installed. It may, for example, default to a baseURL of <http://localhost:8080/baseR4/>.
 
 `fhirServer.username | password` must be changed from defaults in HAPI.
 
@@ -170,9 +173,9 @@ This syntax works for deeper nested config as well. For example, let's configure
 
 The syntax to change the field from **openhim-password** to **newPassword** would be `mediator__api__password=newPassword`
 
-### ElasticSearch Configuration
+### OpenSearch/ElasticSearch Configuration
 
-For ES, the relationship between patient resources in FHIR and what fields are synchronized in ES must be explicitly defined. This is termed the Report Relationship mapping. One must define what resource to be used (patient) and what fields need to be available in ES. After this, the Client Registry reads these fields, and populates ES with the information.
+For OpenSearch/ElasticSearch, the relationship between patient resources in FHIR and what fields are synchronized in OpenSearch/ElasticSearch must be explicitly defined. This is termed the Report Relationship mapping. One must define what resource to be used (patient) and what fields need to be available in OpenSearch/ElasticSearch. After this, the Client Registry reads these fields, and populates OpenSearch/ElasticSearch with the information.
 
 In `resources/Relationships/PatientRelationship.json` there is a template for configuration.
 

docs/admin/installation.md

@@ -25,6 +25,7 @@ The only required dependency is Java >= 8 (1.8).
 See [HAPI FHIR CLI](https://smilecdr.com/hapi-fhir/docs/tools/hapi_fhir_cli.html) for instructions for the OS of choice.
 
 The Client Registry requires FHIR version R4 and HAPI must be started for this version. To run HAPI:
+
 ```
 hapi-fhir-cli run-server -v r4
 ```
@@ -34,45 +35,76 @@ The HAPI Web Testing UI is available at [http://localhost:8080/](http://localhos
 The FHIR Base URL is at [http://localhost:8080/baseR4/](http://localhost:8080/baseR4/)
 
 Visit [http://localhost:8080/](http://localhost:8080/) to ensure HAPI is up and running or
+
 ```sh
 curl -X GET "localhost:8080/baseR4/Patient?"
 ```
 
-## ElasticSearch
+## Indexing
+
+openCR supports both elasticsearch and opensearch, you are free to use either of them but we do recommend opensearch because of elasticsearch license restrictions. Follow instructions below to either install opensearch or elasticsearch
+
+### OpenSearch
+
+Install and start opensearch for the intended OS. See the [install instructions here](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/)
+
+The required version is >=2.1.
+
+The phonetic analysis package must be installed. For example:
+
+```
+/usr/share/opensearch/bin/opensearch-plugin install analysis-phonetic
+```
+
+The string similarity plugin must be installed. See: <https://github.com/DigitalSQR/record-linkage/releases>
+
+Once installed and started, ensure that opensearch is up and running:
+
+```sh
+curl -X GET "localhost:9200/_cat/health?v&pretty"
+```
+
+Status should be yellow for a single-node cluster.
+
+### ElasticSearch
 
 Install and start ES for the intended OS. See the [ES install instructions](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html)
 
 The required version is >=7.6.
 
 The phonetic analysis package must be installed. For example:
+
 ```
 /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-phonetic
 ```
 
-The string similarity plugin must be installed. See: https://github.com/intrahealth/similarity-scoring
-
+The string similarity plugin must be installed. See: <https://github.com/intrahealth/similarity-scoring>
 
 Once installed and started, ensure that ES is up and running:
+
 ```sh
 curl -X GET "localhost:9200/_cat/health?v&pretty"
 ```
-Status should be yellow for a single-node cluster.
 
+Status should be yellow for a single-node cluster.
 
 ## OpenCR Service and UI
 
 Clone the repository into a directory of choice.
+
 ```
 git clone https://github.com/intrahealth/client-registry.git
 ```
 
 Enter the server directory, install node packages.
+
 ```
 cd client-registry/server
 npm install
 ```
 
 Copy and edit the configuration file to your liking.
+
 ```
 cp config/config_development_template.json config/config_development.json
 # edit the servers...
@@ -83,14 +115,15 @@ The minimum changes to start a running standalone system are:
 * Change `fhirServer.baseURL` to "http://localhost:8080/baseR4/"
 
 Run the server from inside client-registry/server:
+
 ```
 # from client-registry/server
 sudo NODE_ENV=development node lib/app.js
 ```
 
 * Visit the UI at: [https://localhost:3000/crux](https://localhost:3000/crux)
-    * **Default username**: [email protected] 
-    * **Default password**: intrahealth
+  * **Default username**: [email protected]
+  * **Default password**: intrahealth
 
 OpenCR may require access to /var/log for logging. This requirement may be changed in the future.
 

docs/admin/installation_full.md

@@ -1,17 +1,24 @@
 # Server Installation
 
 !!! caution
-    Installing and maintaining a production installation is not trivial. This installation method requires strong familiarity with the command line and expertise administering Linux environments. 
+    Installing and maintaining a production installation is not trivial. This installation method requires strong familiarity with the command line and expertise administering Linux environments.
 
 The core production stack consists of four components:
 
-* **OpenCR Service**: This includes primary API for fielding requests, and the record viewing and matching breaking UI. 
+* **OpenCR Service**: This includes primary API for fielding requests, and the record viewing and matching breaking UI.
 * **HAPI FHIR Server + Database**: HAPI FHIR Server is the reference implementation of FHIR in Java. It requries a database backend (e.g. Postgres or MySQL).
+
+Either
+
+* **OpenSearch**: Version >=2.1.0 supported and the analysis-phonetic plugin is required.
+
+OR
+
 * **ElasticSearch**: Version >=7.5 supported and the analysis-phonetic plugin is required.
 
 Optional components:
 
-* **OpenHIM core and OpenHIM admin console**. Requires MongoDB. OpenHIM is an authentication, authorization, and auditing layer. While OpenHIM is optional, nodes and users must be managed in some application if not OpenHIM. Nodes must have certificates issued to query OpenCR and they must be rotated out over time. The OpenCR Service can manage simply installations but using an enterprise secrets management tool is recommended. 
+* **OpenHIM core and OpenHIM admin console**. Requires MongoDB. OpenHIM is an authentication, authorization, and auditing layer. While OpenHIM is optional, nodes and users must be managed in some application if not OpenHIM. Nodes must have certificates issued to query OpenCR and they must be rotated out over time. The OpenCR Service can manage simply installations but using an enterprise secrets management tool is recommended.
 
 ## Prerequisites
 
@@ -30,40 +37,56 @@ HAPI FHIR must use a database backend in production. HAPI FHIR stores the patien
 * The ES integration is separate from HAPI FHIR Server, so there is not need to use it as an indexer. ES only works with an old version of ES.
 * Install and configure the preferred database. Postgres has been tested by the maintainers but any database should work that HAPI supports. Change default passwords on the database.
 * Database replication should be encrypted.
-* Confirm that HAPI accepts requests. 
+* Confirm that HAPI accepts requests.
 * The web interface for HAPI should be disabled for privacy reasons.
 
 !!! caution
     In production, Postgres should run on multiple nodes with replication. This is to ensure high availability and backups of the data.
 
+## OpenSearch
+
+* Follow the instructions for [installation](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/)
+* Systemd is the preferred system and service manager. There are commands to initiate systemd and journalctl.
+* The phonetic analysis package must be installed.
+
+```
+/usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-phonetic
+```
+
+* The string similarity plugin must be installed. See: <https://github.com/DigitalSQR/record-linkage/releases>
 
 ## ElasticSearch
 
 * Follow the instructions for [installation](https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html)
 * Systemd is the preferred system and service manager. There are commands to initiate systemd and journalctl.
 * The phonetic analysis package must be installed.
+
 ```
 /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-phonetic
 ```
-* The string similarity plugin must be installed. See: https://github.com/intrahealth/similarity-scoring
+
+* The string similarity plugin must be installed. See: <https://github.com/intrahealth/similarity-scoring>
 
 !!! caution
     ES is not production-ready when run as one single node. It is recommended to run ES on several nodes. Those nodes can also run followers of Postgres.
 
 ## OpenCR Service and UI
 
 Clone the repository into a directory of choice.
+
 ```
 git clone https://github.com/intrahealth/client-registry.git
 ```
 
 Enter the server directory, install node packages.
+
 ```
 cd client-registry/server
 npm install
 ```
 
 Copy and edit the configuration file to your liking.
+
 ```
 cp config/config_development_template.json config/config_development.json
 # edit the servers...
@@ -74,6 +97,7 @@ The minimum changes to start a running standalone system are:
 * Change `fhirServer.baseURL` to "http://localhost:8080/baseR4/"
 
 Run the server from inside client-registry/server:
+
 ```
 node lib/app.js
 ```
@@ -84,6 +108,6 @@ OpenHIM supports the last 2 versions of NodeJS LTS and requires MongoDB.
 
 * Follow the [instructions](http://openhim.org/docs/installation/npm) to install OpenHIM core and admin console. The maintainers use the NPM PPA installation method.
 * Note the important step to obtain a certificate immediately after installation. The configuration should be that any client must have a certificate and the server has a certificate (mutual TLS).
-* Follow the instructions including console configuration. 
+* Follow the instructions including console configuration.
 * Note the important step to change the console password. It is also recommended that the console only be accessible on a local subnet and not to the WAN.
-* The config mediator.register must be set to true for the OpenCR Service to use OpenHIM.
+* The config mediator.register must be set to true for the OpenCR Service to use OpenHIM.

docs/admin/proficiencies.md

@@ -5,17 +5,17 @@
 OpenCR is not one application, it's several and is expected to be run on Linux. Persons installing and managing OpenCR require advanced expertise with Linux. Here are the major topics where knowledge is required:
 
 * **Linux users and groups**: This includes understanding and restricting `sudo` access.
-* **Networking**: Limiting the public and LAN exposure of services. For instance, HAPI FHIR Server and ElasticSearch should only be exposed to localhost while the UI may be exposed to a LAN subnet, if at all.
-* **Databases**: HAPI FHIR Server requires a database backend. For demos, it can be used with an existing temporary datastore, Derby, but this is not appropriate for maintaining data in production. In production, databases should be backed up and those backups tested as suitable artifacts for recovery. 
-* **Process management**: [systemd and the systemctl](https://www.linode.com/docs/quick-answers/linux-essentials/introduction-to-systemctl/) series of commands are recommended for managing the process lifecycle, including restarting services and logging their status. 
+* **Networking**: Limiting the public and LAN exposure of services. For instance, HAPI FHIR Server and OpenSearch/ElasticSearch should only be exposed to localhost while the UI may be exposed to a LAN subnet, if at all.
+* **Databases**: HAPI FHIR Server requires a database backend. For demos, it can be used with an existing temporary datastore, Derby, but this is not appropriate for maintaining data in production. In production, databases should be backed up and those backups tested as suitable artifacts for recovery.
+* **Process management**: [systemd and the systemctl](https://www.linode.com/docs/quick-answers/linux-essentials/introduction-to-systemctl/) series of commands are recommended for managing the process lifecycle, including restarting services and logging their status.
 * **Logging**: Suitable logging practice requires safely logging the minimum data required to understand performance and uptime.
 * **Auditing**: The OpenCR software stack should be regularly audited. The security section discusses the range of issues to address.
 
 ## Software
 
 For installing and managing an existing OpenCR installation, there are a handful of commands that can be learned.
 
-* **Java**: HAPI FHIR Server and ElasticSearch are written in Java. Java applications are generally built with frameworks for common design patterns and often built using Gradle or Maven.
+* **Java**: HAPI FHIR Server and OpenSearch/ElasticSearch are written in Java. Java applications are generally built with frameworks for common design patterns and often built using Gradle or Maven.
 
 * **JavaScript and Node**: The OpenCR Service is written in Node, a popular JavaScript framework for building RESTful applications. Node applications are packaged around the Node Package Manager.