-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Proof controlling dataflows section (#346)
Refines the "Managing Dataflows" section. Mostly minor adjustments to phrasing and changes to make it read like general documentation instead of a concrete tutorial.
- Loading branch information
Showing
1 changed file
with
23 additions
and
24 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -145,81 +145,80 @@ $> sdf worker shutdown <name> | |
|
||
### Deploying Dataflows to Workers | ||
|
||
Once worker is selected, you can deploy the dataflow using `deploy` command: | ||
Once a worker is selected, you can deploy a dataflow defined in a `dataflow.yaml` file using the `deploy` command: | ||
|
||
```bash | ||
$> sdf deploy | ||
``` | ||
|
||
The deploy command is similar to the run command. It deploys the dataflow and starts the REPL prompt. In deploy mode, the CLI sends the request to the worker. If no worker is selected, an error message will be displayed. | ||
The deploy command is similar to the run command. It deploys a dataflow and starts the REPL prompt. In deploy mode, the CLI sends requests to the worker. If no worker is selected, an error message will be displayed. | ||
|
||
```bash | ||
Error: No workers. run `sdf worker create` to create one. | ||
``` | ||
|
||
When you are running dataflow in the worker, it will indicate name of the worker in the prompt: | ||
When you are running a dataflow on a worker, it will indicate the name of the worker in the prompt: | ||
|
||
```bash | ||
$> sdf deploy | ||
[main] >> show state | ||
[jolly-pond] >> show state | ||
``` | ||
|
||
### Listing and selecting dataflow | ||
### Listing and Selecting Dataflows | ||
|
||
To list all dataflows running in the worker, you can use the `show dataflow` command which shows the fully qualified name of the dataflow and its status. | ||
To list all dataflows running in the worker, you can use the `show dataflow` command which shows the fully qualified name of each dataflow and its status. | ||
|
||
```bash | ||
$> sdf deploy | ||
[jolly-pond]>> show dataflow | ||
Dataflow Status Last Updated | ||
myorg/wordcount-simple@0.10 running 2 days ago | ||
myorg/wordcount-simple@0.1.0 running 2 days ago | ||
* myorg/[email protected] running 10 minutes ago | ||
[jolly-pond]>> | ||
``` | ||
|
||
Other commands like `show state` requires active dataflow. If there is no active dataflow, it will show error message. | ||
Other commands like `show state` require an active dataflow. If there is no active dataflow, it will show an error message. | ||
|
||
```bash | ||
[jolly-pond]>> show state | ||
No dataflow selected. Run `select dataflow` | ||
[jolly-pond]>> | ||
``` | ||
|
||
To select the dataflow, you can use `dataflow select` with the fully qualified dataflow name. | ||
To select a dataflow, you can use the `dataflow select` command with the fully qualified dataflow name. | ||
|
||
```bash | ||
[jolly-pond]>> select dataflow myorg/wordcount-simple@0.10 | ||
dataflow switched to: myorg/wordcount-simple@0.10 | ||
[jolly-pond]>> select dataflow myorg/wordcount-simple@0.1.0 | ||
dataflow switched to: myorg/wordcount-simple@0.1.0 | ||
``` | ||
### Stopping and Restarting dataflow | ||
### Stopping and Restarting Dataflows | ||
In certain cases, you want to stop the dataflow but not delete it. You can use the `stop` command. | ||
In certain cases, you may want to stop a dataflow without deleting it. This can be done with the `stop` command. | ||
```bash | ||
[jolly-pond]>> stop dataflow myorg/[email protected] | ||
[jolly-pond]>> stop dataflow myorg/[email protected] | ||
Stopped dataflow: `my-org/[email protected]` | ||
``` | ||
And restart: | ||
You can then restart the dataflow with the `restart` command. | ||
```bash | ||
[jolly-pond]>> restart dataflow myorg/[email protected] | ||
[jolly-pond]>> restart dataflow myorg/[email protected] | ||
Restarted dataflow: `my-org/[email protected]` | ||
``` | ||
Note that `stop` is not persistent. If worker is restarted, the dataflow will be restarted. | ||
### Deleting dataflow | ||
Note that the `stop` command is not persistent. If a worker is restarted, its dataflows will be restarted as well. | ||
To delete the dataflow, you can use the `dataflow delete` command. | ||
### Deleting Dataflows | ||
After you delete the dataflow, it will no longer be listed in the dataflow list. | ||
To delete a dataflow, you can use the `dataflow delete` command. After you delete a dataflow, it will no longer appear in the dataflow list. | ||
```bash | ||
[jolly-pond]>> delete dataflow myorg/wordcount-simple@0.10 | ||
[jolly-pond]>> delete dataflow myorg/wordcount-simple@0.1.0 | ||
Dataflow Status Last Updated | ||
* myorg/[email protected] running 10 minutes ago | ||
``` | ||
Note that since `myorg/[email protected] ` is deleted, it is no longer listed in the dataflow list. | ||
[SDF CLI]: sdf/cli/index.mdx | ||
[CLI]: sdf/cli/index.mdx |