Skip to content

Commit

Permalink
Merge branch 'risingwavelabs:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
agiron123 authored Apr 19, 2024
2 parents 3b2219a + 44d9c62 commit 3e25370
Show file tree
Hide file tree
Showing 16 changed files with 202 additions and 180 deletions.
68 changes: 68 additions & 0 deletions cloud/PrivateLink-create-a-connection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
id: PrivateLink-create-a-connection
title: Create a PrivateLink connection
description: Create a PrivateLink connection.
slug: /create-a-connection
---

Follow the steps below to create a PrivateLink connection between RisingWave Cloud and your VPC.

## Prerequisites

- You need to create a cluster with the Pro plan or Enterprise plan in RisingWave Cloud:

- See [Choose a cluster plan](/cluster-choose-a-cluster-plan.md) for more information. Please note that Developer clusters do not support PrivateLink connections.

- The VPC you want to connect to and your cluster must be in the same region. If your preferred region is not available when creating a cluster, contact our [support team](mailto:[email protected]) or [sales team](mailto:[email protected]).

- You need to set up a PrivateLink service in your VPC and make sure it runs properly. The following links might be helpful:

- For AWS, see [Share your services through AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html).
- For GCP, see [GCP Published services](https://cloud.google.com/vpc/docs/about-vpc-hosted-services).
- For Azure, see [Azure Private Link services](https://learn.microsoft.com/en-us/azure/private-link/private-link-service-overview).

:::note
Azure Private Link integration is currently in development and will be available soon.
:::

## Steps

1. Go to the [**PrivateLink**](https://cloud.risingwave.com/connection/) page and click **Create PrivateLink**.

2. For **Platform**, select your cloud service provider. Currently, RisingWave Cloud supports **AWS** PrivateLink and **GCP** Private Service Connect.

3. For **Cluster**, select the cluster you want to connect the VPC to. Ensure that the VPC and the cluster are in the same region.

4. For **Name name**, enter a descriptive name for the connection.

5. For **Endpoint service name** or **Service attachment**:

<details><summary>If you choose AWS as the platform, enter the service name of the endpoint service.</summary>

You can find it in the [Amazon VPC console](https://console.aws.amazon.com/vpc/)**Endpoint services****Service name** section.

<img
src={require('./images/aws-endpoint-service-name.png').default}
alt="AWS endpoint service name"
/>

</details>

<details><summary>If you choose GCP as the platform, enter the server target URL of the service attachment.</summary>

You can find it in the [Google Cloud Console](https://console.cloud.google.com/)**Network services****Private Service Connect**.

<img
src={require('./images/gcp-service-attachment.png').default}
alt="GCP Service attachment"
/>

</details>

6. Click **Confirm** to create the connection.

## What's next

Now, you can create a source or sink with the PrivateLink connection using SQL.

For details on how to use the VPC endpoint to create a source with the PrivateLink connection, see [Create source with PrivateLink connection](/docs/current/ingest-from-kafka/#create-source-with-privatelink-connection); for creating a sink, see [Create sink with PrivateLink connection](/docs/current/create-sink-kafka/#create-sink-with-privatelink-connection).
12 changes: 12 additions & 0 deletions cloud/PrivateLink-drop-a-connection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
id: PrivateLink-drop-a-connection
title: Drop a PrivateLink connection
description: If you no longer need to connect to a PrivateLink, you can drop the connection.
slug: /drop-a-connection
---

Follow the steps below to drop a connection to your VPC when you no longer need it.

1. Go to the [**Connection**](https://cloud.risingwave.com/connection/) page and click **Create PrivateLink**.

2. Hover over the connection you want to drop and click the delete button, then confirm the deletion.
63 changes: 63 additions & 0 deletions cloud/PrivateLink-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
id: PrivateLink-overview
title: PrivateLink connection
description: Manage PrivateLink connections.
slug: /PrivateLink-overview
---

## Concept

In RisingWave Cloud, if you want to connect RisingWave clusters with your services inside your private Virtual Private Cloud (VPC) network, you can use the PrivateLink service to establish a private and secure connection between RisingWave Cloud and your private VPC in the same region.

RisingWave Cloud utilizes the the private connection capability of the underlying Cloud vendors to establish the PrivateLink connection. In particular, the PrivateLink service is built on top of the following services:

- [AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html)
- [GCP Private Service Connect](https://cloud.google.com/vpc/docs/private-service-connect)
- [Azure Private Link](https://learn.microsoft.com/en-us/azure/private-link/)

:::note
Azure Private Link integration is currently in development and will be available soon.
:::

The diagram below depicts a high-level overview of how PrivateLink service works. Both all three platforms share the same pattern of network structure so that you can configure them in the same way automatically.

<img
src={require('./images/PrivateLink-diagram.png').default}
alt="VPC connection diagram"
/>

On the **RisingWave Cloud** side, RisingWave Cloud will create an endpoint (specifically an AWS VPC endpoint, GCP Private Service Connect endpoint, or Azure private endpoint) and bind it with one running RisingWave cluster.

On the **Customer** side, you need to set up a PrivateLink service (specifically an AWS endpoint service, GCP published service, or Azure Private Link service) in your VPC network first.

<grid
container
direction="row"
spacing="15"
justifyContent="space-between"
justifyItems="stretch"
alignItems="stretch">

<grid item xs={12} sm={6} md={6}>

<card
title="Create a connection"
content="Create a PrivateLink connection between RisingWave Cloud and your VPC."
cloud="create-a-connection"
style={{height: "80%"}}
/>

</grid>

<grid item xs={12} sm={6} md={6}>

<card
title="Drop a connection"
content="If you no longer need to connect to a VPC, you can drop the connection."
cloud="drop-a-connection"
style={{height: "80%"}}
/>

</grid>

</grid>
Binary file added cloud/images/PrivateLink-diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed cloud/images/vpc-diagram.png
Binary file not shown.
84 changes: 0 additions & 84 deletions cloud/vpc-create-a-connection.md

This file was deleted.

18 changes: 0 additions & 18 deletions cloud/vpc-drop-a-connection.md

This file was deleted.

60 changes: 0 additions & 60 deletions cloud/vpc-overview.md

This file was deleted.

6 changes: 3 additions & 3 deletions docs/get-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ risingwave
Ensure [Docker Desktop](https://docs.docker.com/get-docker/) is installed and running in your environment.

```shell
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.7.0-standalone single_node
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:latest single_node
```

### Homebrew
Expand All @@ -47,7 +47,7 @@ Ensure [Homebrew](https://brew.sh/) is installed, and run the following commands

```shell
brew tap risingwavelabs/risingwave
brew install risingwave@1.7-standalone
brew install risingwave
risingwave
```

Expand Down Expand Up @@ -151,4 +151,4 @@ Congratulations! You've successfully started RisingWave and conducted some initi
* [Example A: Ingest data from Kafka](https://github.com/risingwavelabs/awesome-stream-processing/blob/main/00-get-started/01-ingest-kafka-data.md)
* [Example B: Ingest data from Postgres CDC](https://github.com/risingwavelabs/awesome-stream-processing/blob/main/00-get-started/02-ingest-pg-cdc.md)
- See [this GitHub directory](https://github.com/risingwavelabs/risingwave/tree/main/integration_tests) for ready-to-run demos and integration examples.
- Read our documentation to learn about how to ingest data from data streaming sources, transform data, and deliver data to downstream systems.
- Read our documentation to learn about how to ingest data from data streaming sources, transform data, and deliver data to downstream systems.
4 changes: 2 additions & 2 deletions docs/guides/create-sink-kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,11 +214,11 @@ FORMAT PLAIN ENCODE JSON;

```

## Create sink with VPC connection
## Create sink with PrivateLink connection

If your Kafka sink service is located in a different VPC from RisingWave, use AWS PrivateLink or GCP Private Service Connect to establish a secure and direct connection. For details on how to set up an AWS PrivateLink connection, see [Create an AWS PrivateLink connection](/sql/commands/sql-create-connection.md#create-an-aws-privatelink-connection).

To create a Kafka sink with a VPC connection, in the WITH section of your `CREATE SINK` statement, specify the following parameters.
To create a Kafka sink with a PrivateLink connection, in the WITH section of your `CREATE SINK` statement, specify the following parameters.

|Parameter| Notes|
|---|---|
Expand Down
Loading

0 comments on commit 3e25370

Please sign in to comment.