Skip to content

Commit

Permalink
[DRAFT]For Test and Review: React component work all merged in single…
Browse files Browse the repository at this point in the history
… PR. (#1992)

* Kedro viz as react component documentation

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Viz doc added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Standalone Example Repository section added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* save file details added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Section rearrangement

Signed-off-by: Jitendra Gundaniya <[email protected]>

* README.md file path is changed for PyPI

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Restoring existing README.md to root

Signed-off-by: Jitendra Gundaniya <[email protected]>

* works

* sidebar display

* Code review suggestion

Signed-off-by: Jitendra Gundaniya <[email protected]>

* preview removed

Signed-off-by: Jitendra Gundaniya <[email protected]>

* metadataPanel prop added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* test fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Release note added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Update RELEASE.md

Co-authored-by: rashidakanchwala <[email protected]>
Signed-off-by: Jitendra Gundaniya <[email protected]>

* toggleNodeClick added back

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Parent div element as wrapper added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Update README.md

Co-authored-by: Tynan DeBold <[email protected]>
Signed-off-by: Jitendra Gundaniya <[email protected]>

* Update README.npm.md

Co-authored-by: Tynan DeBold <[email protected]>
Signed-off-by: Jitendra Gundaniya <[email protected]>

* Generic actions for a redux state update

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Refactor code

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Alignment fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Test fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* test fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* test fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Update src/actions/index.js

Co-authored-by: rashidakanchwala <[email protected]>
Signed-off-by: Jitendra Gundaniya <[email protected]>

* Variable renaming

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Updating url on node click

Signed-off-by: Jitendra Gundaniya <[email protected]>

* updated readme for npm

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Annotation image updated with metadataPanel

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Annotation image url updated in npm readme.

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Missing react import added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* props replace by options

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Props replaced by options

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Release note added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Release note added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Code review suggestions

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Rebuild

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Fix doc build error

Signed-off-by: Jitendra Gundaniya <[email protected]>

* comment removed

Signed-off-by: Jitendra Gundaniya <[email protected]>

* code block type changed to jsx

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Release note updated

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Code review suggestion added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Stateful url updates on props change

Signed-off-by: Jitendra Gundaniya <[email protected]>

* annotation image update

Signed-off-by: Jitendra Gundaniya <[email protected]>

* UPDATE_STATE_FROM_OPTIONS test added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Minimap prop refactor

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Props updated

Signed-off-by: Jitendra Gundaniya <[email protected]>

* kedro viz annotation updates

Signed-off-by: Jitendra Gundaniya <[email protected]>

* propTypes added

Signed-off-by: Jitendra Gundaniya <[email protected]>

* pipelineBtn renamed to expandPipelinesBtn

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Moving zoomToolbar into display

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Refactor

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* globalToolbar renamed to globalNavigation

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Test fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Docs updated as pre new refactor work

Signed-off-by: Jitendra Gundaniya <[email protected]>

* annotation image updated

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Test fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* README.md conflict resolved

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Doc copy changes.

* Fix build fail

* Code review fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Test fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Code review fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Code review fix

Signed-off-by: Jitendra Gundaniya <[email protected]>

* Rearrange props in alphabetical order

Signed-off-by: Jitendra Gundaniya <[email protected]>

* file rename

Signed-off-by: Jitendra Gundaniya <[email protected]>

---------

Signed-off-by: Jitendra Gundaniya <[email protected]>
Signed-off-by: Jitendra Gundaniya <[email protected]>
Co-authored-by: rashidakanchwala <[email protected]>
Co-authored-by: rashidakanchwala <[email protected]>
Co-authored-by: Tynan DeBold <[email protected]>
  • Loading branch information
4 people authored Jul 22, 2024
1 parent faceed9 commit 6534297
Show file tree
Hide file tree
Showing 34 changed files with 564 additions and 197 deletions.
Binary file added .github/img/kedro_viz_annotation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ jobs:
run: |-
node -v
make build
# Renaming README.npm.md to README.md just for npm
- name: Use NPM specific README
run: mv README.npm.md README.md

- name: Publish to npm
run: npm publish
Expand Down
218 changes: 218 additions & 0 deletions README.npm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
# Kedro-Viz

<br />
<p align="center">

![Kedro-Viz Pipeline Visualisation](https://raw.githubusercontent.com/kedro-org/kedro-viz/main/.github/img/banner.png)

</p>

<p align="center">
✨ <em> Data Science Pipelines. Beautifully Designed</em> ✨
<br />
Live Demo: <a href="https://demo.kedro.org/" target="_blank">https://demo.kedro.org/</a>
</p>

<br />

[![CircleCI](https://circleci.com/gh/kedro-org/kedro-viz/tree/main.svg?style=shield)](https://circleci.com/gh/kedro-org/kedro-viz/tree/main)
[![Documentation](https://readthedocs.org/projects/kedro/badge/?version=stable)](https://docs.kedro.org/en/stable/visualisation/)
[![Python Version](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11-orange.svg)](https://pypi.org/project/kedro-viz/)
[![PyPI version](https://img.shields.io/pypi/v/kedro-viz.svg?color=yellow)](https://pypi.org/project/kedro-viz/)
[![Downloads](https://static.pepy.tech/badge/kedro-viz/week)](https://pepy.tech/project/kedro-viz)
[![npm version](https://img.shields.io/npm/v/@quantumblack/kedro-viz.svg?color=cc3534)](https://badge.fury.io/js/%40quantumblack%2Fkedro-viz)
[![License](https://img.shields.io/badge/license-Apache%202.0-3da639.svg)](https://opensource.org/licenses/Apache-2.0)
[![code style: prettier](https://img.shields.io/badge/code_style-prettier-ff69b4.svg)](https://github.com/prettier/prettier)
[![Slack Organisation](https://img.shields.io/badge/slack-chat-blueviolet.svg?label=Kedro%20Slack&logo=slack)](https://slack.kedro.org)

## Introduction

Kedro-Viz is an interactive development tool for building data science pipelines with [Kedro](https://github.com/kedro-org/kedro). Kedro-Viz also allows users to view and compare different runs in the Kedro project.

## Features

- ✨ Complete visualisation of a Kedro project and its pipelines
- 🎨 Supports light & dark themes out of the box
- 🚀 Scales to big pipelines with hundreds of nodes
- 🔎 Highly interactive, filterable and searchable
- 🔬 Focus mode for modular pipeline visualisation
- 📊 Rich metadata side panel to display parameters, plots, and more.
- 📊 Supports all types of [Plotly charts](https://plotly.com/javascript/)
- ♻️ Autoreload on code change
- 🧪 Supports tracking and comparing runs in a Kedro project
- 🎩 Many more to come

## Installation

- As a standalone React component (for embedding Kedro-Viz in your web application).

To install the standalone React component:

```bash
npm install @quantumblack/kedro-viz
```

## Usage

To use Kedro-Viz as a standalone React component, you can follow the example below. Please note that Kedro-Viz does not support server-side rendering (SSR). If you are using Next.js or another SSR framework, you should be aware of this limitation.

```javascript
import KedroViz from '@quantumblack/kedro-viz';
import '@quantumblack/kedro-viz/lib/styles/styles.min.css';

const MyApp = () => (
<div style={{height: `100vh`}}>
<KedroViz
data={json}
options={/* Options to configure Kedro Viz */}
/>
</div>
);
```

To use with NextJS:

```javascript
import '@quantumblack/kedro-viz/lib/styles/styles.min.css';
import dynamic from 'next/dynamic';

const NoSSRKedro = dynamic(() => import('@quantumblack/kedro-viz'), {
ssr: false,
});

const MyApp = () => <NoSSRKedro data={json} />;
```

The JSON can be obtained by running:

```bash
kedro viz run --save-file=<path-to-save-kedro-viz-data-to-a-directory>
```

On successful execution of the command above, it will generate a folder at the specified directory, containing the following structure:

```
<filename>/api/
├── main
├── nodes
│ ├── 23c94afb
│ ├── 28754fab
│ ├── 2ab3579f
│ ├── 329e963c
│ ├── 369acf98
│ └── ...
└── pipelines
├── __default__
├── data_processing
├── data_science
└── ...
```

Use the `main` file as the input JSON for the `data` prop in your Kedro-Viz component.

We also recommend wrapping the `Kedro-Viz` component with a parent HTML/JSX element that has a specified height (as seen in the above example) in order for Kedro-Viz to be styled properly.

**_Our documentation contains [additional examples on how to visualise with Kedro-Viz.](https://docs.kedro.org/en/stable/visualisation/index.html)_**

## Configure Kedro-viz with `options`

The example below demonstrates how to configure your kedro-viz using different `options`.

```
<KedroViz
data={json}
options={{
display: {
expandPipelinesBtn: true,
exportBtn: true,
globalNavigation: true,
labelBtn: true,
layerBtn: true,
metadataPanel: true,
miniMap: true,
sidebar: true,
zoomToolbar: true,
},
expandAllPipelines: false,
nodeType: {
disabled: {parameters: true}
},
tag: {
enabled: {companies: true}
},
theme: "dark"
}}
/>
```

| Name | Type | Default | Description |
| ------------ | ------- | ------- | ----------- |
| `data` | `{ edges: array (required), layers: array, nodes: array (required), tags: array }` | - | Pipeline data to be displayed on the chart |
| options.display | | | |
| `expandPipelinesBtn` | boolean | true | Show/Hide expand pipelines button |
| `exportBtn` | boolean | true | Show/Hide export button |
| `globalNavigation` | boolean | true | Show/Hide global navigation |
| `labelBtn` | boolean | true | Show/Hide label button |
| `layerBtn` | boolean | true | Show/Hide layer button |
| `metadataPanel` | boolean | true | Show/Hide Metadata Panel |
| `miniMap` | boolean | true | Show/Hide Mini map and mini map button |
| `sidebar` | boolean | true | Show/Hide Sidebar and action toolbar |
| `zoomToolbar` | boolean | true | Show/Hide zoom-in, zoom-out and zoom reset buttons together |
| options.expandAllPipelines | boolean | false | Expand/Collapse Modular pipelines on first load |
| options.nodeType | `{disabled: {parameters: boolean,task: boolean,data: boolean}}` | `{disabled: {parameters: true,task: false,data: false}}` | Configuration for node type options |
| options.tag | `{enabled: {<tagName>: boolean}}` | - | Configuration for tag options |
| options.theme | string | dark | select `Kedro-Viz` theme : dark/light |


### Note
When `display.sidebar` is `false`, `display.miniMap` prop will be ignored.

All components are annotated to understand their positions in the Kedro-Viz UI.

![Kedro-Viz component annotation](https://raw.githubusercontent.com/kedro-org/kedro-viz/main/.github/img/kedro_viz_annotation.png)


## Standalone Example Repository

We have created a [kedro-viz-standalone](https://github.com/kedro-org/kedro-viz-standalone.git) repository to demonstrate how to use Kedro-Viz in standalone mode or embedded in a React application.

This repository provides a fully functional example of Kedro-Viz, showcasing how to integrate it into your projects. The example includes setup instructions and demonstrates various features of Kedro-Viz.

To get started, clone the repository, install the dependencies and run the example React application:

```
git clone https://github.com/kedro-org/kedro-viz-standalone.git
cd kedro-viz-standalone
npm install
npm start
```

## Feature Flags

Kedro-Viz uses feature flags to roll out some experimental features. We have the following flags -

| Flag | Description |
| ------------------ | --------------------------------------------------------------------------------------- |
| sizewarning | From release v3.9.1. Show a warning before rendering very large graphs (default `true`) |
| expandAllPipelines | From release v4.3.2. Expand all modular pipelines on first load (default `false`) |

To enable or disable a flag, click on the settings icon in the toolbar and toggle the flag on/off.

Kedro-Viz also logs a message in your browser's [developer console](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/What_are_browser_developer_tools#The_JavaScript_console) to show the available flags and their values as currently set on your machine.

## Maintainers

Kedro-Viz is maintained by the [Kedro team](https://docs.kedro.org/en/stable/contribution/technical_steering_committee.html#kedro-maintainers) and a number of [contributors from across the world](https://github.com/kedro-org/Kedro-Viz/contributors).

## Contribution

If you want to contribute to Kedro-Viz, please check out our [contributing guide](./CONTRIBUTING.md).

## License

Kedro-Viz is licensed under the [Apache 2.0](https://github.com/kedro-org/kedro-viz/blob/main/LICENSE.md) License.

## Citation

If you're an academic, Kedro-Viz can also help you, for example, as a tool to visualise how your publication's pipeline is structured. Find our citation reference on [Zenodo](https://doi.org/10.5281/zenodo.4277218).
3 changes: 3 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,10 @@ Please follow the established format:
- Enable/disable preview for all the datasets when publishing Kedro-Viz from UI. (#1895)
- Display published URLs. (#1907)
- Conditionally move session store and stats file to .viz directory. (#1915)
- Introduce `metadataPanel` in `display` prop in Kedro-Viz react component. (#1965)
- Refactor namespace pipelines. (#1897)
- Expose the internal Redux state through `options` prop while using Kedro-Viz as a React component. (#1969)
- Enhance documentation for the Kedro-Viz standalone React component. (#1954)

## Bug fixes and other changes

Expand Down
15 changes: 14 additions & 1 deletion docs/source/kedro-viz_visualisation.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,4 +225,17 @@ For example, to launch Kedro Viz on a specific host and port with autoreload ena
```

![Open your project's Kedro Viz inside a new tab](./images/run_viz_with_args.png)
![Kedro Viz inside a new tab](./images/viz_running_in_new_tab.png)
![Kedro Viz inside a new tab](./images/viz_running_in_new_tab.png)

## As a standalone React component

To use Kedro-Viz as a standalone React component (for embedding Kedro-Viz in your web application), you can follow the example below:

```jsx
import KedroViz from '@quantumblack/kedro-viz';
import '@quantumblack/kedro-viz/lib/styles/styles.min.css';

const MyApp = () => <div style={{height: `100vh`}}><KedroViz data={json} options={options} /></div>;
```

For more information on how to use kedro as React component with all possible props please refer to [Kedro-Viz on npm](https://www.npmjs.com/package/@quantumblack/kedro-viz)
11 changes: 10 additions & 1 deletion package/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,14 @@ To use Kedro-Viz as a standalone React component, you can follow the example bel
import KedroViz from '@quantumblack/kedro-viz';
import '@quantumblack/kedro-viz/lib/styles/styles.min.css';
const MyApp = () => <KedroViz data={json} />;
const MyApp = () => (
<div style={{height: `100vh`}}>
<KedroViz
data={json}
options={/* Options to configure Kedro Viz */}
/>
</div>
);
```
To use with NextJS:
Expand All @@ -261,6 +268,8 @@ kedro viz run --save-file=filename
We also recommend wrapping the `Kedro-Viz` component with a parent HTML/JSX element that has a specified height (as seen in the above example) in order for Kedro-Viz to be styled properly.
For more information on how to use Kedro-Viz as a React component, follow this [guide](https://www.npmjs.com/package/@quantumblack/kedro-viz)
**_Our documentation contains [additional examples on how to visualise with Kedro-Viz.](https://docs.kedro.org/en/stable/visualisation/index.html)_**
## Feature Flags
Expand Down
13 changes: 13 additions & 0 deletions src/actions/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -331,3 +331,16 @@ export function updateRunNotes(notes, runId) {
runId,
};
}

export const UPDATE_STATE_FROM_OPTIONS = 'UPDATE_STATE_FROM_OPTIONS';

/**
* Update state with latest options prop coming from the react component
* @param {Object} updatedOptions
*/
export const updateStateFromOptions = (updatedOptions) => {
return {
type: UPDATE_STATE_FROM_OPTIONS,
payload: updatedOptions,
};
};
9 changes: 3 additions & 6 deletions src/actions/pipelines.js
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,7 @@ export function loadInitialPipelineData() {
const url = getUrl('main');
// obtain the status of expandAllPipelines to decide whether it needs to overwrite the
// list of visible nodes
const expandAllPipelines =
state.display.expandAllPipelines || state.expandAllPipelines;
const expandAllPipelines = state.expandAllPipelines;
let newState = await loadJsonData(url).then((data) =>
preparePipelineState(data, true, expandAllPipelines)
);
Expand All @@ -122,7 +121,7 @@ export function loadInitialPipelineData() {
*/
export function loadPipelineData(pipelineID) {
return async function (dispatch, getState) {
const { dataSource, pipeline, display, expandAllPipelines } = getState();
const { dataSource, pipeline, expandAllPipelines } = getState();

if (pipelineID && pipelineID === pipeline.active) {
return;
Expand All @@ -136,10 +135,8 @@ export function loadPipelineData(pipelineID) {
active: pipelineID,
});

const shouldExpandAllPipelines =
display.expandAllPipelines || expandAllPipelines;
const newState = await loadJsonData(url).then((data) =>
preparePipelineState(data, false, shouldExpandAllPipelines)
preparePipelineState(data, false, expandAllPipelines)
);

// Set active pipeline here rather than dispatching two separate actions,
Expand Down
Loading

0 comments on commit 6534297

Please sign in to comment.