Skip to content

Commit

Permalink
Merge branch 'main' into doc/slicing-pipeline
Browse files Browse the repository at this point in the history
Signed-off-by: Huong Nguyen <huongg1409@gmail>
  • Loading branch information
Huong Nguyen committed Aug 22, 2024
2 parents 0626edc + d171b50 commit b26c691
Show file tree
Hide file tree
Showing 9 changed files with 315 additions and 5 deletions.
4 changes: 3 additions & 1 deletion README.npm.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ The example below demonstrates how to configure your kedro-viz using different `
| Name | Type | Default | Description |
| ------------ | ------- | ------- | ----------- |
| `data` | `{ edges: array (required), layers: array, nodes: array (required), tags: array }` | - | Pipeline data to be displayed on the chart |
| `onActionCallback` | function | - | Callback function to be invoked when the specified action is dispatched. e.g. `const action = { type: NODE_CLICK, payload: node }; onActionCallback(action);` |
| options.display | | | |
| `expandPipelinesBtn` | boolean | true | Show/Hide expand pipelines button |
| `exportBtn` | boolean | true | Show/Hide export button |
Expand All @@ -166,7 +167,8 @@ The example below demonstrates how to configure your kedro-viz using different `


### Note
When `display.sidebar` is `false`, `display.miniMap` prop will be ignored.
- `onActionCallback` callback is only called when the user clicks on a node in the flowchart, and we are passing the node object as the payload in the callback argument. In future releases, we will add more actions to be dispatched in this callback.
- When `display.sidebar` is `false`, `display.miniMap` prop will be ignored.

All components are annotated to understand their positions in the Kedro-Viz UI.

Expand Down
4 changes: 4 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ Please follow the established format:
-->
# Upcoming Release

## Major features and improvements

- Introduce `onActionCallback` prop in Kedro-Viz react component. (#2022)

## Bug fixes and other changes

- Fixes design issues in metadata panel. (#2009)
Expand Down
162 changes: 162 additions & 0 deletions docs/source/cli-docs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# Kedro Viz CLI reference

The Kedro Viz CLI provides commands to visualise Kedro pipelines, deploy them to cloud platforms, and export the visualisation data. Below is a detailed description of the available commands and options.

## Commands

### `kedro viz`

Launches a local Kedro Viz instance to visualise a Kedro pipeline.

**Usage:**

```bash
kedro viz [OPTIONS]
```

**Description:**

This command launches the Kedro Viz server to visualise a Kedro pipeline. It is functionally the same as `kedro viz run`. If no sub-command is provided, `run` is used by default.

**Options:**

This command accepts all the options that are available in the `kedro viz`, `kedro viz run` command. See the `kedro viz run` section for a complete list of options.

### `kedro viz run`

Launches a local Kedro Viz instance to visualise a Kedro pipeline.

**Usage:**

```bash
kedro viz run [OPTIONS]
```

**Options:**

- `--host <host>`
- Host that Kedro Viz will listen to. Defaults to `localhost`.

- `--port <port>`
- TCP port that Kedro Viz will listen to. Defaults to `4141`.

- `--browser / --no-browser`
- Whether to open the Kedro Viz interface in the default browser. The browser will open if the host is `localhost`. Defaults to `True`.

- `--load-file <path>`
- Path to load Kedro Viz data from a directory. If provided, Kedro Viz will load the visualisation data from this path instead of generating it from the pipeline.

- `--save-file <path>`
- Path to save Kedro Viz data to a directory. If provided, the visualisation data will be saved to this path for later use.

- `--pipeline, -p <pipeline>`
- Name of the registered pipeline to visualise. If not set, the default pipeline is visualised.

- `--env, -e {environment>}`
- Kedro configuration environment. If not specified, the catalog config in `local` will be used. You can also set this through the `KEDRO_ENV` environment variable.

- `--autoreload, -a`
- Enable autoreload of the Kedro Viz server when a Python or YAML file changes in the Kedro project.

- `--include-hooks`
- Include all registered hooks in the Kedro project for visualisation.

- `--params <params>`
- Specify extra parameters for the Kedro Viz run. This option supports the same format as the `params` option in the Kedro CLI.


```{note}
When running Kedro Viz locally with the `--autoreload` option, the server will automatically restart whenever there are changes to Python, YAML, or JSON files in the Kedro project. This is particularly useful during development.
```


### `kedro viz deploy`

Deploy and host Kedro Viz on a specified cloud platform.

```{note}
The `deploy` command supports deployment to AWS, Azure and GCP. Ensure that your cloud credentials and configurations are correctly set up before deploying.
```

**Usage:**

```bash
kedro viz deploy [OPTIONS]
```

**Options:**

- `--platform <platform>`
- The cloud platform to host Kedro Viz on. Supported platforms include `aws` `azure` and `gcp`. This option is required.

- `--endpoint <endpoint>`
- The static website hosted endpoint. This option is required.

- `--bucket-name <bucket-name>`
- The name of the bucket where Kedro Viz will be hosted. This option is required.

- `--include-hooks`
- Include all registered hooks in the Kedro project in the deployed visualisation.

- `--include-previews`
- Include previews for all datasets in the deployed visualisation.

### `kedro viz build`

Create a build directory of a local Kedro Viz instance with Kedro project data.

**Usage:**

```bash
kedro viz build [OPTIONS]
```

**Options:**

- `--include-hooks`
- Include all registered hooks in the Kedro project in the built visualisation.

- `--include-previews`
- Include previews for all datasets in the built visualisation.


## Examples

### Running Kedro Viz locally

To run Kedro Viz on your local machine, use:

```bash
kedro viz
```

To specify a particular pipeline and environment:

```bash
kedro viz -p my_pipeline -e dev
```

or

```bash
kedro viz run -p my_pipeline -e dev
```

### Deploying Kedro Viz to AWS

To deploy Kedro Viz to an S3 bucket on AWS:

```bash
kedro viz deploy --platform aws --endpoint http://mybucket.s3-website-us-west-2.amazonaws.com --bucket-name mybucket
```

### Building Kedro Viz to host on multiple platforms

To create a build directory with the visualisation data:

```bash
kedro viz build --include-previews
```



2 changes: 1 addition & 1 deletion docs/source/experiment_tracking.md
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ Parallel coordinates displays all metrics on a single graph, with each vertical
When in comparison view, comparing runs highlights your selections on the respective chart types, improving readability even in the event there is a multitude of data points.

```{note}
The following graphic is taken from the [Kedro-Viz experiment tracking demo](https://demo.kedro.org/experiment-tracking) (it is not a visualisation from the example code you created above).
The following graphic is taken from the [Kedro-Viz experiment tracking demo](https://demo.kedro.org/) (it is not a visualisation from the example code you created above).
```

![](./images/experiment-tracking-metrics-comparison.gif)
Expand Down
7 changes: 7 additions & 0 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,10 @@ preview_datasets
slice_a_pipeline
experiment_tracking
```

```{toctree}
:caption: Other Documentation
:maxdepth: 1
cli-docs
```
6 changes: 5 additions & 1 deletion src/components/app/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,11 @@ class App extends React.Component {
constructor(props) {
super(props);
const initialState = getInitialState(props);
this.store = configureStore(initialState, this.props.data);
this.store = configureStore(
initialState,
this.props.data,
this.props.onActionCallback
);
}

componentDidMount() {
Expand Down
49 changes: 49 additions & 0 deletions src/store/middleware.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import { TOGGLE_NODE_CLICKED } from '../actions/nodes';

/**
* Middleware to handle custom callback actions in the Redux store.
*
* This middleware intercepts actions dispatched to the Redux store and checks for the action type.
* If the action type matches and callback function is provided,
* Then it invokes the provided callback function with type and payload.
*
* @param {Function} callback - The callback function to be invoked when the specified action is dispatched.
* @returns {Function} - A middleware function to be used in the Redux store.
*/
const createCallbackMiddleware =
(callback) => (store) => (next) => (action) => {
if (!callback) {
return next(action);
}

switch (action.type) {
case TOGGLE_NODE_CLICKED:
if (action.nodeClicked) {
const state = store.getState();
const node = state?.graph?.nodes?.find(
(node) => node.id === action.nodeClicked
);
if (node) {
const nodeClickAction = {
type: TOGGLE_NODE_CLICKED,
payload: node,
};
callback(nodeClickAction);
}
}
break;
/**
* Add additional cases here to handle other action types.
* Ensure on whatever action you want to trigger, It should be a Redux action.
* And payload should be in Redux state.
* Example:
* const action = { type: SLICE_PIPELINE, payload: runCommand };
callback(action);
*/
default:
break;
}
return next(action);
};

export default createCallbackMiddleware;
74 changes: 74 additions & 0 deletions src/store/middleware.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
import createCallbackMiddleware from './middleware';

describe('createCallbackMiddleware', () => {
let store;
let next;
let callback;
let middleware;

beforeEach(() => {
store = {
getState: jest.fn(),
};
next = jest.fn();
callback = jest.fn();
middleware = createCallbackMiddleware(callback)(store)(next);
});

it('should call the callback with the correct node when action type is TOGGLE_NODE_CLICKED and node is found', () => {
const nodeClicked = '123';
const node = { id: nodeClicked, name: 'Node 123' };
const nodeClickAction = { type: 'TOGGLE_NODE_CLICKED', payload: node };
const action = { type: 'TOGGLE_NODE_CLICKED', nodeClicked };

store.getState.mockReturnValue({
graph: {
nodes: [node],
},
});

middleware(action);

expect(callback).toHaveBeenCalledWith(nodeClickAction);
expect(next).toHaveBeenCalledWith(action);
});

it('should not call the callback if action type is different', () => {
const action = { type: 'OTHER_ACTION', nodeClicked: '123' };

middleware(action);

expect(callback).not.toHaveBeenCalled();
expect(next).toHaveBeenCalledWith(action);
});

it('should not call the callback if node is not found', () => {
const action = { type: 'TOGGLE_NODE_CLICKED', nodeClicked: '123' };

store.getState.mockReturnValue({
graph: {
nodes: [{ id: '456', name: 'Node 456' }],
},
});

middleware(action);

expect(callback).not.toHaveBeenCalled();
expect(next).toHaveBeenCalledWith(action);
});

it('should not call the callback if callback is not provided', () => {
middleware = createCallbackMiddleware(null)(store)(next);
const action = { type: 'TOGGLE_NODE_CLICKED', nodeClicked: '123' };

store.getState.mockReturnValue({
graph: {
nodes: [{ id: '123', name: 'Node 123' }],
},
});

middleware(action);

expect(next).toHaveBeenCalledWith(action);
});
});
12 changes: 10 additions & 2 deletions src/store/store.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { getGraphInput } from '../selectors/layout';
import { calculateGraph } from '../actions/graph';
import { saveLocalStorage, pruneFalseyKeys } from './helpers';
import { localStorageName, localStorageRunsMetadata } from '../config';
import createCallbackMiddleware from './middleware';

/**
* Watch the getGraphInput selector, and dispatch an asynchronous action to
Expand Down Expand Up @@ -70,15 +71,22 @@ const saveStateToLocalStorage = (state) => {
* Configure initial state and create the Redux store
* @param {Object} initialState Initial Redux state (from initial-state.js)
* @param {Object} dataType type of pipeline data - "static" or "json" (if data is loaded from API)
* @param {Function} onActionCallback Callback function to be called on specific action
* @return {Object} Redux store
*/
export default function configureStore(initialState, dataType) {
export default function configureStore(
initialState,
dataType,
onActionCallback
) {
const composeEnhancers =
window.__REDUX_DEVTOOLS_EXTENSION_COMPOSE__ || compose;

const callbackMiddleware = createCallbackMiddleware(onActionCallback);
const store = createStore(
reducer,
initialState,
composeEnhancers(applyMiddleware(thunk))
composeEnhancers(applyMiddleware(thunk, callbackMiddleware))
);

// dispatch the calculateGraph action to ensure the graph nodes still gets rendered
Expand Down

0 comments on commit b26c691

Please sign in to comment.