Skip to content

Commit

Permalink
Proof controlling dataflows section (#346)
Browse files Browse the repository at this point in the history
Refines the "Managing Dataflows" section. 

Mostly minor adjustments to phrasing and changes to make it read like
general documentation instead of a concrete tutorial.
  • Loading branch information
crajcan authored Jan 30, 2025
1 parent 77441bc commit 660320b
Showing 1 changed file with 23 additions and 24 deletions.
47 changes: 23 additions & 24 deletions sdf/deployment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -145,81 +145,80 @@ $> sdf worker shutdown <name>

### Deploying Dataflows to Workers

Once worker is selected, you can deploy the dataflow using `deploy` command:
Once a worker is selected, you can deploy a dataflow defined in a `dataflow.yaml` file using the `deploy` command:

```bash
$> sdf deploy
```

The deploy command is similar to the run command. It deploys the dataflow and starts the REPL prompt. In deploy mode, the CLI sends the request to the worker. If no worker is selected, an error message will be displayed.
The deploy command is similar to the run command. It deploys a dataflow and starts the REPL prompt. In deploy mode, the CLI sends requests to the worker. If no worker is selected, an error message will be displayed.

```bash
Error: No workers. run `sdf worker create` to create one.
```

When you are running dataflow in the worker, it will indicate name of the worker in the prompt:
When you are running a dataflow on a worker, it will indicate the name of the worker in the prompt:

```bash
$> sdf deploy
[main] >> show state
[jolly-pond] >> show state
```

### Listing and selecting dataflow
### Listing and Selecting Dataflows

To list all dataflows running in the worker, you can use the `show dataflow` command which shows the fully qualified name of the dataflow and its status.
To list all dataflows running in the worker, you can use the `show dataflow` command which shows the fully qualified name of each dataflow and its status.

```bash
$> sdf deploy
[jolly-pond]>> show dataflow
Dataflow Status Last Updated
myorg/wordcount-simple@0.10 running 2 days ago
myorg/wordcount-simple@0.1.0 running 2 days ago
* myorg/[email protected] running 10 minutes ago
[jolly-pond]>>
```

Other commands like `show state` requires active dataflow. If there is no active dataflow, it will show error message.
Other commands like `show state` require an active dataflow. If there is no active dataflow, it will show an error message.

```bash
[jolly-pond]>> show state
No dataflow selected. Run `select dataflow`
[jolly-pond]>>
```

To select the dataflow, you can use `dataflow select` with the fully qualified dataflow name.
To select a dataflow, you can use the `dataflow select` command with the fully qualified dataflow name.

```bash
[jolly-pond]>> select dataflow myorg/wordcount-simple@0.10
dataflow switched to: myorg/wordcount-simple@0.10
[jolly-pond]>> select dataflow myorg/wordcount-simple@0.1.0
dataflow switched to: myorg/wordcount-simple@0.1.0
```
### Stopping and Restarting dataflow
### Stopping and Restarting Dataflows
In certain cases, you want to stop the dataflow but not delete it. You can use the `stop` command.
In certain cases, you may want to stop a dataflow without deleting it. This can be done with the `stop` command.
```bash
[jolly-pond]>> stop dataflow myorg/[email protected]
[jolly-pond]>> stop dataflow myorg/[email protected]
Stopped dataflow: `my-org/[email protected]`
```
And restart:
You can then restart the dataflow with the `restart` command.
```bash
[jolly-pond]>> restart dataflow myorg/[email protected]
[jolly-pond]>> restart dataflow myorg/[email protected]
Restarted dataflow: `my-org/[email protected]`
```
Note that `stop` is not persistent. If worker is restarted, the dataflow will be restarted.
### Deleting dataflow
Note that the `stop` command is not persistent. If a worker is restarted, its dataflows will be restarted as well.
To delete the dataflow, you can use the `dataflow delete` command.
### Deleting Dataflows
After you delete the dataflow, it will no longer be listed in the dataflow list.
To delete a dataflow, you can use the `dataflow delete` command. After you delete a dataflow, it will no longer appear in the dataflow list.
```bash
[jolly-pond]>> delete dataflow myorg/wordcount-simple@0.10
[jolly-pond]>> delete dataflow myorg/wordcount-simple@0.1.0
Dataflow Status Last Updated
* myorg/[email protected] running 10 minutes ago
```
Note that since `myorg/[email protected] ` is deleted, it is no longer listed in the dataflow list.
[SDF CLI]: sdf/cli/index.mdx
[CLI]: sdf/cli/index.mdx

0 comments on commit 660320b

Please sign in to comment.