diff --git a/ojdbc-provider-aws/README.md b/ojdbc-provider-aws/README.md index 6f92d368..9db97063 100644 --- a/ojdbc-provider-aws/README.md +++ b/ojdbc-provider-aws/README.md @@ -1,7 +1,150 @@ # Oracle JDBC Providers for AWS -This module contains providers for integration between Oracle JDBC and AWS. +This module contains providers for integration between Oracle JDBC and +Amazon Web Services (AWS). ## Centralized Config Providers -Finish me \ No newline at end of file +
+
AWS S3 Configuration +Provider
+
Provides connection properties managed by the S3 service
+
AWS Secrets Manager Configuration +Provider
+
Provides connection properties managed by the Secrets Manager service
+
+ +Visit any of the links above to find information and usage examples for a +particular provider. + +## Installation + +All providers in this module are distributed as single jar on the Maven Central +Repository. The jar is compiled for JDK 8, and is forward compatible with later +JDK versions. The coordinates for the latest release are: +```xml + + com.oracle.database.jdbc + ojdbc-provider-aws + 1.2.0 + +``` + +## Authentication + +Providers use AWS SDK which supports +[Default credentials provider chain](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials-chain.html), +which looks for credentials in a set of defined locations and use those +credentials to authenticate requests to AWS. + +The default credentials provider chain searches for credentials in one of the following locations using a predefined sequence: + +1. Java system properties +2. Environment variables +3. Web identity token from AWS Security Token Service +4. The shared credentials and config files +5. Amazon ECS container credentials +6. Amazon EC2 instance IAM role-provided credentials + +## AWS S3 Config Provider +The Oracle DataSource uses a new prefix `jdbc:oracle:thin:@config-awss3:` to be able to identify that the configuration parameters should be loaded using AWS S3. Users only need to indicate the S3 URI of the object that contains the JSON payload, with the following syntax: + +
+jdbc:oracle:thin:@config-awass3://{s3-uri}
+
+ +### JSON Payload format + +There are 3 fixed values that are looked at the root level. + +- connect_descriptor (required) +- user (optional) +- password (optional) + +The rest are dependent on the driver, in our case `/jdbc`. The key-value pairs that are with sub-prefix `/jdbc` will be applied to a DataSource. The key values are constant keys which are equivalent to the properties defined in the [OracleConnection](https://docs.oracle.com/en/database/oracle/oracle-database/23/jajdb/oracle/jdbc/OracleConnection.html) interface. + +For example, let's suppose an url like: + +
+jdbc:oracle:thin:@config-awss3://s3://mybucket/payload_ojdbc_objectstorage.json
+
+ +And the JSON Payload for the file **payload_ojdbc_objectstorage.json** in **mybucket** as following: + +```json +{ + "connect_descriptor": "(description=(retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1521)(host=adb.us-phoenix-1.oraclecloud.com))(connect_data=(service_name=xsxsxs_dbtest_medium.adb.oraclecloud.com))(security=(ssl_server_dn_match=yes)))", + "user": "scott", + "password": { + "type": "awssecretsmanager", + "value": "test-secret", + "key_name": "db-password" + }, + "jdbc": { + "oracle.jdbc.ReadTimeout": 1000, + "defaultRowPrefetch": 20, + "autoCommit": "false" + } +} +``` + +The sample code below executes as expected with the previous configuration. + +```java + OracleDataSource ds = new OracleDataSource(); + ds.setURL("jdbc:oracle:thin:@config-awss3://s3://mybucket/payload_ojdbc_objectstorage.json"); + Connection cn = ds.getConnection(); + Statement st = cn.createStatement(); + ResultSet rs = st.executeQuery("select sysdate from dual"); + if (rs.next()) + System.out.println("select sysdate from dual: " + rs.getString(1)); +``` + +### Password JSON Object + +For the JSON type of provider (AWS S3, AWS Secrets Manager, HTTP/HTTPS, File) the password is an object itself with the following spec: + +- type + - Mandatory + - Possible values + - ocivault + - azurevault + - base64 + - gcpsecretmanager + - awssecretsmanager +- value + - Mandatory + - Possible values + - OCID of the secret (if ocivault) + - Azure Key Vault URI (if azurevault) + - Base64 Encoded password (if base64) + - GCP resource name (if gcpsecretmanager) + - AWS Secret name (if awssecretsmanager) + - Text +- key_name + - Optional + - Possible values + - Name of the key, if stored as key-value pairs in AWS Secrets Manager +- authentication + - Optional + - Possible Values + - method + - optional parameters (depends on the cloud provider). + +## AWS Secrets Manager Config Provider +Apart from AWS S3, users can also store JSON Payload in the content of AWS Secrets Manager secret. Users need to indicate the secret name: + +
+jdbc:oracle:thin:@config-awssecretsmanager://{secret-name}
+
+ +The JSON Payload retrieved by AWS Secrets Manager Provider follows the same format in [AWS S3 Configuration Provider](#json-payload-format). + +## Caching configuration + +Config providers in this module store the configuration in caches to minimize +the number of RPC requests to remote location. See +[Caching configuration](../ojdbc-provider-azure/README.md#caching-configuration) for more +details of the caching mechanism. + + diff --git a/ojdbc-provider-aws/example-test.properties b/ojdbc-provider-aws/example-test.properties new file mode 100644 index 00000000..519715c8 --- /dev/null +++ b/ojdbc-provider-aws/example-test.properties @@ -0,0 +1,88 @@ +################################################################################ +# Copyright (c) 2024 Oracle and/or its affiliates. +# +# The Universal Permissive License (UPL), Version 1.0 +# +# Subject to the condition set forth below, permission is hereby granted to any +# person obtaining a copy of this software, associated documentation and/or data +# (collectively the "Software"), free of charge and under any and all copyright +# rights in the Software, and any and all patent rights owned or freely +# licensable by each licensor hereunder covering either (i) the unmodified +# Software as contributed to or provided by such licensor, or (ii) the Larger +# Works (as defined below), to deal in both +# +# (a) the Software, and +# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +# one is included with the Software (each a "Larger Work" to which the Software +# is contributed by such licensors), +# +# without restriction, including without limitation the rights to copy, create +# derivative works of, display, perform, and distribute the Software and make, +# use, sell, offer for sale, import, export, have made, and have sold the +# Software and the Larger Work(s), and to sublicense the foregoing rights on +# either these or other terms. +# +# This license is subject to the following condition: +# The above copyright notice and either this complete permission notice or at +# a minimum a reference to the UPL must be included in all copies or +# substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. +################################################################################ + +# This file provides examples of properties that configure tests in this +# module. +# +# QUICK GUIDE TO RUNNING TESTS: +# 1. Create a copy of this file named "test.properties": +# cp example-test.properties test.properties +# 2. In test.properties, replace example values with real values (the file is +# .gitignore'd, so sensitive info won't be checked in to the repo) +# 3. Comment out any lines for which a value can not be provided (tests are +# skipped if no value is configured). +# 4. mvn clean verify +# +# CONFIGURING TEST PROPERTIES +# Test properties are read from a properties file by the TestProperties class. +# The TestProperties class can be found in: +# ojdbc-provider-common/src/testFixtures/java/oracle/jdbc/provider/TestProperties.java +# The default behavior of TestProperties is to read a file named +# "test.properties" in the current directory. A non-default location may be +# specified as a JVM system property: +# mvn clean verify -Doracle.jdbc.provider.TestProperties=/path/to/my-test.properties +# +# MAINTAINING THIS FILE +# Project maintainers should add an example to this file anytime they write a +# test which requires a new property. Not doing so will inflict pain and +# suffering upon our fellow programmers, and will also lead to increased +# maintenance costs. +# +# IGNORING UNCONFIGURED PROPERTIES +# No test should cause a build failure due to an unconfigured property. +# Using JUnit terminology: A test should "abort" rather than "fail" when a +# property is not configured. This means that the test does not pass, but it +# does not cause the build to fail either. +# Methods of the TestProperties class will automatically abort a test if a +# property is not configured. The org.junit.jupiter.api.Assumptions class may +# also be used directly to abort a test. +# There is NO environment in which ALL tests can be run. Some tests may +# require authentication as a managed identity in an Azure VM, while other +# tests require authentication as an instance principal in an OCI compute +# instance; These environments are mutually exclusive. This is one reason why +# tests can not fail the build if a required property is not set. +# A more practical reason is that developers may not need to run all tests if +# their changes are isolated to single module. For instance, a developer +# working on an OCI provider should not need to set up an Azure tenancy to test +# their changes. + +# The URI of an AWS S3. +AWS_S3_URI=s3://{your-s3-ari} + +# The name of an AWS Secret that contains JSON payload +AWS_SECRET_NAME=your-secret-name diff --git a/ojdbc-provider-aws/pom.xml b/ojdbc-provider-aws/pom.xml index 1ee9ddd9..ffbddb0d 100644 --- a/ojdbc-provider-aws/pom.xml +++ b/ojdbc-provider-aws/pom.xml @@ -55,6 +55,13 @@ software.amazon.awssdk secretsmanager + + + com.oracle.database.jdbc + ojdbc-provider-common + tests + test-jar + org.junit.jupiter junit-jupiter-api @@ -63,10 +70,5 @@ org.junit.jupiter junit-jupiter-engine - - com.oracle.database.security - oraclepki - test - diff --git a/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/configuration/AwsJsonSecretsManagerProvider.java b/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/configuration/AwsJsonSecretsManagerProvider.java index a24900fa..ff38cbca 100644 --- a/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/configuration/AwsJsonSecretsManagerProvider.java +++ b/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/configuration/AwsJsonSecretsManagerProvider.java @@ -15,8 +15,9 @@ public class AwsJsonSecretsManagerProvider @Override public char[] getSecret(OracleJsonObject jsonObject) { ParameterSet parameterSet = - PARAMETER_SET_PARSER.parseNamedValues( - JsonSecretUtil.toNamedValues(jsonObject)); + PARAMETER_SET_PARSER + .parseNamedValues( + JsonSecretUtil.toNamedValues(jsonObject)); String secretString = SecretsManagerFactory.getInstance() .request(parameterSet) diff --git a/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/configuration/AwsSecretsManagerConfigurationProvider.java b/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/configuration/AwsSecretsManagerConfigurationProvider.java index a9caeeb3..dee11197 100644 --- a/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/configuration/AwsSecretsManagerConfigurationProvider.java +++ b/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/configuration/AwsSecretsManagerConfigurationProvider.java @@ -21,7 +21,7 @@ public class AwsSecretsManagerConfigurationProvider extends OracleConfigurationJ ParameterSetParser.builder() .addParameter("value", SecretsManagerFactory.SECRET_NAME) .addParameter("REGION", SecretsManagerFactory.REGION) - .addParameter("key", SecretsManagerFactory.KEY)) + .addParameter("key_name", SecretsManagerFactory.KEY_NAME)) .build(); @Override @@ -41,6 +41,6 @@ public InputStream getJson(String secretId) { @Override public String getType() { - return "awssecret"; + return "awssecretsmanager"; } } diff --git a/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/secrets/SecretsManagerFactory.java b/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/secrets/SecretsManagerFactory.java index 1b2677ea..9b9c1f8e 100644 --- a/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/secrets/SecretsManagerFactory.java +++ b/ojdbc-provider-aws/src/main/java/oracle/jdbc/provider/aws/secrets/SecretsManagerFactory.java @@ -37,7 +37,7 @@ public final class SecretsManagerFactory * The name of the key if the secret contains key-value pairs. * This is an optional parameter. * */ - public static final Parameter KEY = + public static final Parameter KEY_NAME = Parameter.create(); private static final OracleJsonFactory JSON_FACTORY = new OracleJsonFactory(); @@ -65,7 +65,7 @@ public Resource request( String secretName = parameterSet.getRequired(SECRET_NAME); String region = parameterSet.getOptional(REGION); - String key = parameterSet.getOptional(KEY); + String key = parameterSet.getOptional(KEY_NAME); SecretsManagerClientBuilder builder = SecretsManagerClient.builder() .credentialsProvider(() -> awsCredentials); diff --git a/ojdbc-provider-aws/src/test/java/oracle/provider/aws/configuration/AwsS3ConfigurationProviderTest.java b/ojdbc-provider-aws/src/test/java/oracle/provider/aws/configuration/AwsS3ConfigurationProviderTest.java new file mode 100644 index 00000000..103a5501 --- /dev/null +++ b/ojdbc-provider-aws/src/test/java/oracle/provider/aws/configuration/AwsS3ConfigurationProviderTest.java @@ -0,0 +1,36 @@ +package oracle.provider.aws.configuration; + +import oracle.jdbc.provider.TestProperties; +import oracle.jdbc.spi.OracleConfigurationProvider; +import org.junit.jupiter.api.Test; + +import java.sql.SQLException; +import java.util.Properties; + +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class AwsS3ConfigurationProviderTest { + + static { + OracleConfigurationProvider.allowedProviders.add("awss3"); + } + + private static final OracleConfigurationProvider PROVIDER = + OracleConfigurationProvider.find("awss3"); + + /** + * Verifies if AWS S3 Configuration Provider works with default authentication + * @throws SQLException + */ + @Test + public void testDefaultAuthentication() throws SQLException { + String location = + TestProperties.getOrAbort( + AwsTestProperty.AWS_S3_URI); + Properties properties = PROVIDER + .getConnectionProperties(location); + assertTrue(properties.containsKey("URL"), "Contains property URL"); + assertTrue(properties.containsKey("user"), "Contains property user"); + assertTrue(properties.containsKey("password"), "Contains property password"); + } +} diff --git a/ojdbc-provider-aws/src/test/java/oracle/provider/aws/configuration/AwsSecretsManagerConfigurationProviderTest.java b/ojdbc-provider-aws/src/test/java/oracle/provider/aws/configuration/AwsSecretsManagerConfigurationProviderTest.java new file mode 100644 index 00000000..63fcd255 --- /dev/null +++ b/ojdbc-provider-aws/src/test/java/oracle/provider/aws/configuration/AwsSecretsManagerConfigurationProviderTest.java @@ -0,0 +1,36 @@ +package oracle.provider.aws.configuration; + +import oracle.jdbc.provider.TestProperties; +import oracle.jdbc.spi.OracleConfigurationProvider; +import org.junit.jupiter.api.Test; + +import java.sql.SQLException; +import java.util.Properties; + +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class AwsSecretsManagerConfigurationProviderTest { + + static { + OracleConfigurationProvider.allowedProviders.add("awssecretsmanager"); + } + + private static final OracleConfigurationProvider PROVIDER = + OracleConfigurationProvider.find("awssecretsmanager"); + + /** + * Verifies if AWS Secrets Manager Configuration Provider works with default authentication + * @throws SQLException + */ + @Test + public void testDefaultAuthentication() throws SQLException { + String location = + TestProperties.getOrAbort( + AwsTestProperty.AWS_SECRET_NAME); + Properties properties = PROVIDER + .getConnectionProperties(location); + assertTrue(properties.containsKey("URL"), "Contains property URL"); + assertTrue(properties.containsKey("user"), "Contains property user"); + assertTrue(properties.containsKey("password"), "Contains property password"); + } +} diff --git a/ojdbc-provider-aws/src/test/java/oracle/provider/aws/configuration/AwsTestProperty.java b/ojdbc-provider-aws/src/test/java/oracle/provider/aws/configuration/AwsTestProperty.java new file mode 100644 index 00000000..716b58ed --- /dev/null +++ b/ojdbc-provider-aws/src/test/java/oracle/provider/aws/configuration/AwsTestProperty.java @@ -0,0 +1,6 @@ +package oracle.provider.aws.configuration; + +public enum AwsTestProperty { + AWS_S3_URI, + AWS_SECRET_NAME +} diff --git a/ojdbc-provider-gcp/README.md b/ojdbc-provider-gcp/README.md index 9b483a6d..68a1d54f 100644 --- a/ojdbc-provider-gcp/README.md +++ b/ojdbc-provider-gcp/README.md @@ -133,6 +133,7 @@ For the JSON type of provider (GCP Object Storage, HTTP/HTTPS, File) the passwor - azurevault - base64 - gcpsecretmanager + - awssecretsmanager - value - Mandatory - Possible values @@ -140,6 +141,7 @@ For the JSON type of provider (GCP Object Storage, HTTP/HTTPS, File) the passwor - Azure Key Vault URI (if azurevault) - Base64 Encoded password (if base64) - GCP resource name (if gcpsecretmanager) + - AWS Secret name (if awssecretsmanager) - Text - authentication - Optional diff --git a/ojdbc-provider-gcp/pom.xml b/ojdbc-provider-gcp/pom.xml index 117dd492..4feaafab 100644 --- a/ojdbc-provider-gcp/pom.xml +++ b/ojdbc-provider-gcp/pom.xml @@ -46,13 +46,6 @@ google-cloud-storage - - - com.oracle.database.jdbc - ojdbc-provider-common - tests - test-jar - com.oracle.database.jdbc ojdbc-provider-common diff --git a/ojdbc-provider-oci/README.md b/ojdbc-provider-oci/README.md index f53610fe..9e24d21a 100644 --- a/ojdbc-provider-oci/README.md +++ b/ojdbc-provider-oci/README.md @@ -134,12 +134,16 @@ For the JSON type of provider (OCI Object Storage, HTTP/HTTPS, File) the passwor - ocivault - azurevault - base64 + - gcpsecretmanager + - awssecretsmanager - value - Mandatory - Possible values - OCID of the secret (if ocivault) - Azure Key Vault URI (if azurevault) - Base64 Encoded password (if base64) + - GCP resource name (if gcpsecretmanager) + - AWS Secret name (if awssecretsmanager) - Text - authentication - Optional (it will apply defaults in the same way as described in [Configuring Authentication](#configuring-authentication)). diff --git a/ojdbc-provider-samples/src/main/java/oracle/jdbc/provider/aws/configuration/AwsS3Example.java b/ojdbc-provider-samples/src/main/java/oracle/jdbc/provider/aws/configuration/AwsS3Example.java index 65f6fa04..e361c294 100644 --- a/ojdbc-provider-samples/src/main/java/oracle/jdbc/provider/aws/configuration/AwsS3Example.java +++ b/ojdbc-provider-samples/src/main/java/oracle/jdbc/provider/aws/configuration/AwsS3Example.java @@ -50,7 +50,9 @@ public static void main(String[] args) throws SQLException { // Sample default URL if non present if (args.length == 0) { - url = "jdbc:oracle:thin:@config-awss3://s3://{bucket-name}/{key-name}"; +// url = "jdbc:oracle:thin:@config-awss3://s3://{bucket-name}/{key-name}"; + url = "jdbc:oracle:thin:@config-awss3://s3://tinglwan-general-bucket/folder1/payload_ojdbc_adb_aws_secret.json"; +// url = "jdbc:oracle:thin:@config-file:///Users/tinglwang/Notes/ojdbc-plugins/ociobject/payload_ojdbc_wallet_location_base64_sso.json"; } else { url = args[0]; } diff --git a/ojdbc-provider-samples/src/main/java/oracle/jdbc/provider/aws/configuration/AwsSecretsManagerConfigurationExample.java b/ojdbc-provider-samples/src/main/java/oracle/jdbc/provider/aws/configuration/AwsSecretsManagerConfigurationExample.java index 8ffa5ccf..d5ba47e8 100644 --- a/ojdbc-provider-samples/src/main/java/oracle/jdbc/provider/aws/configuration/AwsSecretsManagerConfigurationExample.java +++ b/ojdbc-provider-samples/src/main/java/oracle/jdbc/provider/aws/configuration/AwsSecretsManagerConfigurationExample.java @@ -13,7 +13,7 @@ public static void main(String[] args) throws SQLException { // Sample default URL if non present if (args.length == 0) { - url = "jdbc:oracle:thin:@config-awssecret://{secret-name}"; + url = "jdbc:oracle:thin:@config-awssecretsmanager://{secret-name}"; } else { url = args[0]; }