-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore: release version
0.11.10
documentation
- Loading branch information
Showing
185 changed files
with
19,109 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
{ | ||
"label": "NodeJS SDK", | ||
"position": 40 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,103 @@ | ||
--- | ||
title: Examples | ||
sidebar_position: 20 | ||
--- | ||
|
||
# NodeJS SDK Examples | ||
|
||
* This client uses [`node-bindgen`] to wrap the Rust client. | ||
* It supports most administrator features. | ||
* The blocking calls to Fluvio return promises allowing for async on blocking Fluvio calls. | ||
* The [`PartitionConsumer.createStream`] call returns an [`asyncIterator`] to allow iterating over the stream in a for-loop. | ||
|
||
To see the full docs, visit [our typedoc page]. | ||
|
||
[`node-bindgen`]: https://github.com/infinyon/node-bindgen | ||
[our typedoc page]: https://infinyon.github.io/fluvio-client-node/ | ||
[`PartitionConsumer.createStream`]: https://infinyon.github.io/fluvio-client-node/classes/PartitionConsumer.html#createStream | ||
[`asyncIterator`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of | ||
|
||
|
||
## Example Workflow | ||
|
||
Follow the [installation instructions](/) to run this example. | ||
|
||
```js | ||
/** | ||
* This is an example of a basic Fluvio workflow in Typescript | ||
* | ||
* 1. Establish a connection to the Fluvio cluster | ||
* 2. Create a topic to store data in | ||
* 3. Create a producer and send some bytes | ||
* 4. Create a consumer, and stream the data back | ||
*/ | ||
import Fluvio, { Offset, Record } from "@fluvio/client"; | ||
|
||
const TOPIC_NAME = "hello-node"; | ||
const PARTITION = 0; | ||
|
||
async function createTopic() { | ||
try { | ||
// Connect to the Fluvio cluster | ||
console.log("Connecting client to fluvio"); | ||
await fluvio.connect(); | ||
|
||
// Create admin client; | ||
const admin = await fluvio.admin(); | ||
|
||
// Create topic | ||
console.log("Creating topic"); | ||
await admin.createTopic(TOPIC_NAME); | ||
} catch (ex) { | ||
console.log("Topic already exists", ex); | ||
} | ||
} | ||
|
||
const produce = async () => { | ||
// Connect to the Fluvio cluster | ||
console.log("Connecting client to fluvio"); | ||
await fluvio.connect(); | ||
|
||
// Create a topic producer; | ||
const producer = await fluvio.topicProducer(TOPIC_NAME); | ||
await producer.send("example-key", "Hello World! - Time is " + Date()); | ||
}; | ||
|
||
const consume = async () => { | ||
try { | ||
// Connect to the fluvio cluster referenced in the cli profile. | ||
await fluvio.connect(); | ||
|
||
// Create partition consumer | ||
const consumer = await fluvio.partitionConsumer(TOPIC_NAME, PARTITION); | ||
|
||
console.log("read from the end"); | ||
await consumer.stream(Offset.FromEnd(), async (record: Record) => { | ||
// handle record; | ||
console.log(`Key=${record.keyString()}, Value=${record.valueString()}`); | ||
process.exit(0); | ||
}); | ||
} catch (ex) { | ||
console.log("error", ex); | ||
} | ||
}; | ||
|
||
// Create Fluvio Client Instance | ||
const fluvio = new Fluvio(); | ||
createTopic(); | ||
produce(); | ||
consume(); | ||
``` | ||
|
||
### Run | ||
|
||
```shell | ||
$ npx ts-node example.ts | ||
``` | ||
|
||
## Links to Docs: | ||
- [Connect to Fluvio](https://infinyon.github.io/fluvio-client-node/interfaces/FluvioClient.html#connect) | ||
- [Create a Producer](https://infinyon.github.io/fluvio-client-node/interfaces/FluvioClient.html#topicProducer) | ||
- [Send to Topic](https://infinyon.github.io/fluvio-client-node/classes/TopicProducer.html#send) | ||
- [Get a Consumer](https://infinyon.github.io/fluvio-client-node/interfaces/FluvioClient.html#partitionConsumer) | ||
- [Create a Stream](https://infinyon.github.io/fluvio-client-node/classes/PartitionConsumer.html#createStream) |
46 changes: 46 additions & 0 deletions
46
versioned_docs/version-0.11.10/apis/nodejs/installation.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
--- | ||
title: Installation | ||
sidebar_position: 10 | ||
--- | ||
|
||
## NodeJS SDK Installation Guide | ||
|
||
Install [Node.js](https://nodejs.org/en/) (v**16.11.0** or above) | ||
|
||
We are using [`npm`](https://nodejs.dev/en/learn/an-introduction-to-the-npm-package-manager/) as package manager. | ||
|
||
## Create New Node project for Fluvio development | ||
|
||
Run the following commands to set up your project for development: | ||
|
||
```bash | ||
mkdir fluvio-demo && cd fluvio-demo | ||
npm init -y | ||
npm install -D typescript ts-node @types/node | ||
npm install -S @fluvio/client | ||
``` | ||
|
||
And your `package.json` should look similar to the following: | ||
|
||
```json | ||
{ | ||
"name": "fluvio-demo", | ||
"version": "1.0.0", | ||
"description": "", | ||
"main": "index.js", | ||
"scripts": { | ||
"test": "echo \"Error: no test specified\" && exit 1" | ||
}, | ||
"keywords": [], | ||
"author": "", | ||
"license": "ISC", | ||
"devDependencies": { | ||
"@types/node": "^20.2.5", | ||
"ts-node": "^10.8.1", | ||
"typescript": "^5.3.3" | ||
}, | ||
"dependencies": { | ||
"@fluvio/client": "^0.14.6" | ||
} | ||
} | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
{ | ||
"label": "Python SDK", | ||
"position": 30 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,154 @@ | ||
--- | ||
title: Examples | ||
sidebar_position: 20 | ||
--- | ||
|
||
# Python SDK Examples | ||
|
||
* The Python client [wraps the rust client](https://www.infinyon.com/blog/2021/03/python-client/). | ||
* It currently does not support the administrator features that the rust client does. | ||
* The [PartitionConsumer.stream](https://infinyon.github.io/fluvio-client-python/fluvio.html#PartitionConsumer.stream) returns an object which implements the [python iterator convention](https://wiki.python.org/moin/Iterator) to allow for iterating over the stream in a for-loop. | ||
|
||
To see the full docs, visit our [pdoc page](https://infinyon.github.io/fluvio-client-python/fluvio.html). | ||
## Example Workflow | ||
|
||
Follow the [installation instructions](/) to run this example. | ||
|
||
### Prerequisites | ||
|
||
Create the topic used to produce and consume records: | ||
|
||
```bash | ||
fluvio topic create python-data | ||
``` | ||
|
||
### Login | ||
|
||
Login to Infinyon Cloud using username/password | ||
|
||
```python | ||
from fluvio import cloud | ||
cloud.login(email="[email protected]", password="mypassword") | ||
``` | ||
|
||
You can also use the oauth method to log in. However this is only for interactive sessions. | ||
|
||
```python | ||
from fluvio import cloud | ||
cloud.login(Oauth2=true) | ||
``` | ||
|
||
### Producer | ||
|
||
Create a file called `python-produce.py`: | ||
|
||
```python | ||
#!/usr/bin/env python | ||
from datetime import datetime | ||
from fluvio import Fluvio | ||
|
||
TOPIC_NAME = "python-data" | ||
PARTITION = 0 | ||
|
||
if __name__ == "__main__": | ||
# Connect to cluster | ||
fluvio = Fluvio.connect() | ||
|
||
# Produce 10 records to topic | ||
producer = fluvio.topic_producer(TOPIC_NAME) | ||
for x in range(10): | ||
producer.send_string("{}: timestamp: {}".format(x, datetime.now())) | ||
|
||
# Flush the last entry | ||
producer.flush() | ||
``` | ||
|
||
Let's run the file: | ||
|
||
```shell copy="fl" | ||
$ python python-produce.py | ||
``` | ||
|
||
### Consumer | ||
|
||
Create a file called `python-consume.py`: | ||
|
||
```python | ||
#!/usr/bin/env python | ||
from fluvio import Fluvio, Offset | ||
|
||
TOPIC_NAME = "python-data" | ||
PARTITION = 0 | ||
|
||
if __name__ == "__main__": | ||
# Connect to cluster | ||
fluvio = Fluvio.connect() | ||
|
||
# Consume last 10 records from topic | ||
consumer = fluvio.partition_consumer(TOPIC_NAME, PARTITION) | ||
for idx, record in enumerate( consumer.stream(Offset.from_end(10)) ): | ||
print("{}".format(record.value_string())) | ||
|
||
if idx >= 9: | ||
break | ||
``` | ||
|
||
Let's run the file: | ||
|
||
```shell copy="fl" | ||
$ python python-consume.py | ||
``` | ||
|
||
## Limitations | ||
* Fluvio cluster administration is not supported. | ||
* Python [async](https://docs.python.org/3/library/asyncio.html) is not supported. | ||
|
||
## Example with a SmartModule | ||
|
||
```python | ||
#!/usr/bin/env python | ||
import os | ||
from datetime import datetime | ||
from fluvio import Fluvio, Offset, ConsumerCoonfig | ||
|
||
TOPIC_NAME = "hello-python-smartmodule" | ||
PARTITION = 0 | ||
|
||
# This is an example of a basic Fluvio workflow in Python | ||
# | ||
# 1. Create a topic to store data in via CLI | ||
# 2. Establish a connection to the Fluvio cluster | ||
# 3. Create a producer and send some bytes | ||
# 4. Create a consumer, and stream the data back | ||
if __name__ == "__main__": | ||
# Currently the Python client does not support creating topics | ||
# Using the fluvio CLI | ||
os.popen("fluvio topic create {}".format(TOPIC_NAME)) | ||
|
||
# Connect to cluster | ||
fluvio = Fluvio.connect() | ||
|
||
# Produce to topic | ||
producer = fluvio.topic_producer(TOPIC_NAME) | ||
producer.send_string("Hello World! - Time is: {}".format(datetime.now())) | ||
|
||
# Consume from topic | ||
# We're just going to get the last record | ||
consumer = fluvio.partition_consumer(TOPIC_NAME, PARTITION) | ||
|
||
|
||
# Create a ConsumerConfig using your "uppercase-map" smartmodule | ||
config = ConsumerConfig() | ||
config.smartmodule(name="uppercase-map") | ||
|
||
for record in consumer.stream_with_config(Offset.from_end(0), config): | ||
print("{}".format(record.value_string())) | ||
break | ||
``` | ||
|
||
## Links to Docs: | ||
* [Connect to Fluvio](https://infinyon.github.io/fluvio-client-python/fluvio.html#Fluvio.connect) | ||
* [Get a Producer](https://infinyon.github.io/fluvio-client-python/fluvio.html#Fluvio.topic_producer) | ||
* [Send to Topic](https://infinyon.github.io/fluvio-client-python/fluvio.html#TopicProducer.send) | ||
* [Get a Consumer](https://infinyon.github.io/fluvio-client-python/fluvio.html#Fluvio.partition_consumer) | ||
* [Get a Stream](https://infinyon.github.io/fluvio-client-python/fluvio.html#PartitionConsumer.stream) |
23 changes: 23 additions & 0 deletions
23
versioned_docs/version-0.11.10/apis/python/installation.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
--- | ||
title: Installation | ||
sidebar_position: 10 | ||
--- | ||
|
||
## Python SDK Installation Guide | ||
|
||
## Minimum Python | ||
|
||
The minimum Python supported is 3.6. | ||
* https://wiki.python.org/moin/BeginnersGuide/Download | ||
|
||
## Install Python's package manager `pip` | ||
You will need `pip` to install the `fluvio` package from PyPi | ||
* https://pip.pypa.io/en/stable/installation/ | ||
|
||
## Install [`fluvio`](https://pypi.org/project/fluvio/) with `pip` | ||
|
||
Run the following to install [`fluvio`](https://pypi.org/project/fluvio/) package | ||
|
||
```shell copy="fl" | ||
$ pip install fluvio | ||
``` |
Oops, something went wrong.