Skip to content

Commit

Permalink
Update README.md. Add tests for AWS providers.
Browse files Browse the repository at this point in the history
  • Loading branch information
ting-lan-wang committed Dec 6, 2024
1 parent e902829 commit c52336e
Show file tree
Hide file tree
Showing 14 changed files with 335 additions and 22 deletions.
147 changes: 145 additions & 2 deletions ojdbc-provider-aws/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,150 @@
# Oracle JDBC Providers for AWS

This module contains providers for integration between Oracle JDBC and AWS.
This module contains providers for integration between Oracle JDBC and
Amazon Web Services (AWS).

## Centralized Config Providers

Finish me
<dl>
<dt><a href="#aws-config-config-provider">AWS S3 Configuration
Provider</a></dt>
<dd>Provides connection properties managed by the S3 service</dd>
<dt><a href="#aws-secrets-manager-config-provider">AWS Secrets Manager Configuration
Provider</a></dt>
<dd>Provides connection properties managed by the Secrets Manager service</dd>
</dl>

Visit any of the links above to find information and usage examples for a
particular provider.

## Installation

All providers in this module are distributed as single jar on the Maven Central
Repository. The jar is compiled for JDK 8, and is forward compatible with later
JDK versions. The coordinates for the latest release are:
```xml
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc-provider-aws</artifactId>
<version>1.2.0</version>
</dependency>
```

## Authentication

Providers use AWS SDK which supports
[Default credentials provider chain](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials-chain.html),
which looks for credentials in a set of defined locations and use those
credentials to authenticate requests to AWS.

The default credentials provider chain searches for credentials in one of the following locations using a predefined sequence:

1. Java system properties
2. Environment variables
3. Web identity token from AWS Security Token Service
4. The shared credentials and config files
5. Amazon ECS container credentials
6. Amazon EC2 instance IAM role-provided credentials

## AWS S3 Config Provider
The Oracle DataSource uses a new prefix `jdbc:oracle:thin:@config-awss3:` to be able to identify that the configuration parameters should be loaded using AWS S3. Users only need to indicate the S3 URI of the object that contains the JSON payload, with the following syntax:

<pre>
jdbc:oracle:thin:@config-awass3://{s3-uri}
</pre>

### JSON Payload format

There are 3 fixed values that are looked at the root level.

- connect_descriptor (required)
- user (optional)
- password (optional)

The rest are dependent on the driver, in our case `/jdbc`. The key-value pairs that are with sub-prefix `/jdbc` will be applied to a DataSource. The key values are constant keys which are equivalent to the properties defined in the [OracleConnection](https://docs.oracle.com/en/database/oracle/oracle-database/23/jajdb/oracle/jdbc/OracleConnection.html) interface.

For example, let's suppose an url like:

<pre>
jdbc:oracle:thin:@config-awss3://s3://mybucket/payload_ojdbc_objectstorage.json
</pre>

And the JSON Payload for the file **payload_ojdbc_objectstorage.json** in **mybucket** as following:

```json
{
"connect_descriptor": "(description=(retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1521)(host=adb.us-phoenix-1.oraclecloud.com))(connect_data=(service_name=xsxsxs_dbtest_medium.adb.oraclecloud.com))(security=(ssl_server_dn_match=yes)))",
"user": "scott",
"password": {
"type": "awssecretsmanager",
"value": "test-secret",
"key_name": "db-password"
},
"jdbc": {
"oracle.jdbc.ReadTimeout": 1000,
"defaultRowPrefetch": 20,
"autoCommit": "false"
}
}
```

The sample code below executes as expected with the previous configuration.

```java
OracleDataSource ds = new OracleDataSource();
ds.setURL("jdbc:oracle:thin:@config-awss3://s3://mybucket/payload_ojdbc_objectstorage.json");
Connection cn = ds.getConnection();
Statement st = cn.createStatement();
ResultSet rs = st.executeQuery("select sysdate from dual");
if (rs.next())
System.out.println("select sysdate from dual: " + rs.getString(1));
```

### Password JSON Object

For the JSON type of provider (AWS S3, AWS Secrets Manager, HTTP/HTTPS, File) the password is an object itself with the following spec:

- type
- Mandatory
- Possible values
- ocivault
- azurevault
- base64
- gcpsecretmanager
- awssecretsmanager
- value
- Mandatory
- Possible values
- OCID of the secret (if ocivault)
- Azure Key Vault URI (if azurevault)
- Base64 Encoded password (if base64)
- GCP resource name (if gcpsecretmanager)
- AWS Secret name (if awssecretsmanager)
- Text
- key_name
- Optional
- Possible values
- Name of the key, if stored as key-value pairs in AWS Secrets Manager
- authentication
- Optional
- Possible Values
- method
- optional parameters (depends on the cloud provider).

## AWS Secrets Manager Config Provider
Apart from AWS S3, users can also store JSON Payload in the content of AWS Secrets Manager secret. Users need to indicate the secret name:

<pre>
jdbc:oracle:thin:@config-awssecretsmanager://{secret-name}
</pre>

The JSON Payload retrieved by AWS Secrets Manager Provider follows the same format in [AWS S3 Configuration Provider](#json-payload-format).

## Caching configuration

Config providers in this module store the configuration in caches to minimize
the number of RPC requests to remote location. See
[Caching configuration](../ojdbc-provider-azure/README.md#caching-configuration) for more
details of the caching mechanism.


88 changes: 88 additions & 0 deletions ojdbc-provider-aws/example-test.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
################################################################################
# Copyright (c) 2024 Oracle and/or its affiliates.
#
# The Universal Permissive License (UPL), Version 1.0
#
# Subject to the condition set forth below, permission is hereby granted to any
# person obtaining a copy of this software, associated documentation and/or data
# (collectively the "Software"), free of charge and under any and all copyright
# rights in the Software, and any and all patent rights owned or freely
# licensable by each licensor hereunder covering either (i) the unmodified
# Software as contributed to or provided by such licensor, or (ii) the Larger
# Works (as defined below), to deal in both
#
# (a) the Software, and
# (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
# one is included with the Software (each a "Larger Work" to which the Software
# is contributed by such licensors),
#
# without restriction, including without limitation the rights to copy, create
# derivative works of, display, perform, and distribute the Software and make,
# use, sell, offer for sale, import, export, have made, and have sold the
# Software and the Larger Work(s), and to sublicense the foregoing rights on
# either these or other terms.
#
# This license is subject to the following condition:
# The above copyright notice and either this complete permission notice or at
# a minimum a reference to the UPL must be included in all copies or
# substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
################################################################################

# This file provides examples of properties that configure tests in this
# module.
#
# QUICK GUIDE TO RUNNING TESTS:
# 1. Create a copy of this file named "test.properties":
# cp example-test.properties test.properties
# 2. In test.properties, replace example values with real values (the file is
# .gitignore'd, so sensitive info won't be checked in to the repo)
# 3. Comment out any lines for which a value can not be provided (tests are
# skipped if no value is configured).
# 4. mvn clean verify
#
# CONFIGURING TEST PROPERTIES
# Test properties are read from a properties file by the TestProperties class.
# The TestProperties class can be found in:
# ojdbc-provider-common/src/testFixtures/java/oracle/jdbc/provider/TestProperties.java
# The default behavior of TestProperties is to read a file named
# "test.properties" in the current directory. A non-default location may be
# specified as a JVM system property:
# mvn clean verify -Doracle.jdbc.provider.TestProperties=/path/to/my-test.properties
#
# MAINTAINING THIS FILE
# Project maintainers should add an example to this file anytime they write a
# test which requires a new property. Not doing so will inflict pain and
# suffering upon our fellow programmers, and will also lead to increased
# maintenance costs.
#
# IGNORING UNCONFIGURED PROPERTIES
# No test should cause a build failure due to an unconfigured property.
# Using JUnit terminology: A test should "abort" rather than "fail" when a
# property is not configured. This means that the test does not pass, but it
# does not cause the build to fail either.
# Methods of the TestProperties class will automatically abort a test if a
# property is not configured. The org.junit.jupiter.api.Assumptions class may
# also be used directly to abort a test.
# There is NO environment in which ALL tests can be run. Some tests may
# require authentication as a managed identity in an Azure VM, while other
# tests require authentication as an instance principal in an OCI compute
# instance; These environments are mutually exclusive. This is one reason why
# tests can not fail the build if a required property is not set.
# A more practical reason is that developers may not need to run all tests if
# their changes are isolated to single module. For instance, a developer
# working on an OCI provider should not need to set up an Azure tenancy to test
# their changes.

# The URI of an AWS S3.
AWS_S3_URI=s3://{your-s3-ari}

# The name of an AWS Secret that contains JSON payload
AWS_SECRET_NAME=your-secret-name
12 changes: 7 additions & 5 deletions ojdbc-provider-aws/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,13 @@
<groupId>software.amazon.awssdk</groupId>
<artifactId>secretsmanager</artifactId>
</dependency>
<!-- TEST DEPENDENCIES -->
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc-provider-common</artifactId>
<classifier>tests</classifier>
<type>test-jar</type>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
Expand All @@ -63,10 +70,5 @@
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
</dependency>
<dependency>
<groupId>com.oracle.database.security</groupId>
<artifactId>oraclepki</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,9 @@ public class AwsJsonSecretsManagerProvider
@Override
public char[] getSecret(OracleJsonObject jsonObject) {
ParameterSet parameterSet =
PARAMETER_SET_PARSER.parseNamedValues(
JsonSecretUtil.toNamedValues(jsonObject));
PARAMETER_SET_PARSER
.parseNamedValues(
JsonSecretUtil.toNamedValues(jsonObject));

String secretString = SecretsManagerFactory.getInstance()
.request(parameterSet)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ public class AwsSecretsManagerConfigurationProvider extends OracleConfigurationJ
ParameterSetParser.builder()
.addParameter("value", SecretsManagerFactory.SECRET_NAME)
.addParameter("REGION", SecretsManagerFactory.REGION)
.addParameter("key", SecretsManagerFactory.KEY))
.addParameter("key_name", SecretsManagerFactory.KEY_NAME))
.build();

@Override
Expand All @@ -41,6 +41,6 @@ public InputStream getJson(String secretId) {

@Override
public String getType() {
return "awssecret";
return "awssecretsmanager";
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ public final class SecretsManagerFactory
* The name of the key if the secret contains key-value pairs.
* This is an optional parameter.
* */
public static final Parameter<String> KEY =
public static final Parameter<String> KEY_NAME =
Parameter.create();

private static final OracleJsonFactory JSON_FACTORY = new OracleJsonFactory();
Expand Down Expand Up @@ -65,7 +65,7 @@ public Resource<String> request(

String secretName = parameterSet.getRequired(SECRET_NAME);
String region = parameterSet.getOptional(REGION);
String key = parameterSet.getOptional(KEY);
String key = parameterSet.getOptional(KEY_NAME);

SecretsManagerClientBuilder builder = SecretsManagerClient.builder()
.credentialsProvider(() -> awsCredentials);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
package oracle.provider.aws.configuration;

import oracle.jdbc.provider.TestProperties;
import oracle.jdbc.spi.OracleConfigurationProvider;
import org.junit.jupiter.api.Test;

import java.sql.SQLException;
import java.util.Properties;

import static org.junit.jupiter.api.Assertions.assertTrue;

public class AwsS3ConfigurationProviderTest {

static {
OracleConfigurationProvider.allowedProviders.add("awss3");
}

private static final OracleConfigurationProvider PROVIDER =
OracleConfigurationProvider.find("awss3");

/**
* Verifies if AWS S3 Configuration Provider works with default authentication
* @throws SQLException
*/
@Test
public void testDefaultAuthentication() throws SQLException {
String location =
TestProperties.getOrAbort(
AwsTestProperty.AWS_S3_URI);
Properties properties = PROVIDER
.getConnectionProperties(location);
assertTrue(properties.containsKey("URL"), "Contains property URL");
assertTrue(properties.containsKey("user"), "Contains property user");
assertTrue(properties.containsKey("password"), "Contains property password");
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
package oracle.provider.aws.configuration;

import oracle.jdbc.provider.TestProperties;
import oracle.jdbc.spi.OracleConfigurationProvider;
import org.junit.jupiter.api.Test;

import java.sql.SQLException;
import java.util.Properties;

import static org.junit.jupiter.api.Assertions.assertTrue;

public class AwsSecretsManagerConfigurationProviderTest {

static {
OracleConfigurationProvider.allowedProviders.add("awssecretsmanager");
}

private static final OracleConfigurationProvider PROVIDER =
OracleConfigurationProvider.find("awssecretsmanager");

/**
* Verifies if AWS Secrets Manager Configuration Provider works with default authentication
* @throws SQLException
*/
@Test
public void testDefaultAuthentication() throws SQLException {
String location =
TestProperties.getOrAbort(
AwsTestProperty.AWS_SECRET_NAME);
Properties properties = PROVIDER
.getConnectionProperties(location);
assertTrue(properties.containsKey("URL"), "Contains property URL");
assertTrue(properties.containsKey("user"), "Contains property user");
assertTrue(properties.containsKey("password"), "Contains property password");
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package oracle.provider.aws.configuration;

public enum AwsTestProperty {
AWS_S3_URI,
AWS_SECRET_NAME
}
Loading

0 comments on commit c52336e

Please sign in to comment.