Merge branch 'main' into feature-branch/refactor-node-list

.github/workflows/label-community-issues.yml

@@ -0,0 +1,48 @@
+name: Label Community Issues
+
+on:
+  issues:
+    types:
+      - opened
+
+jobs:
+  label:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check if issue author is a member of Kedro org
+        uses: actions/github-script@v6
+        id: membership
+        with:
+          github-token: ${{ secrets.GH_TAGGING_TOKEN }}
+          result-encoding: string
+          script: |
+
+            try {
+              const result = await github.rest.orgs.getMembershipForUser({
+                       org: "kedro-org",
+                       username: '${{ github.actor }}'
+                    })
+
+              console.log(result?.data?.state)
+              if (result?.data?.state == "active"){
+                console.log("%s: detected as an active member of Kedro org", '${{ github.actor }}')
+                return "member";
+              } else {
+                console.log("%s: not detected as active member of Kedro org", '${{ github.actor }}')
+                return "notMember";
+              }
+
+            } catch (error) {
+              console.log("%s: Error occured and marked user as notMember", '${{ github.actor }}')
+              console.log("Error", error.stack);
+              console.log("Error", error.name);
+              console.log("Error", error.message);
+              return "notMember";
+            }
+
+      - name: Label issue if author is from community
+        if: ${{ steps.membership.outputs.result == 'notMember' }}
+        uses: actions-ecosystem/action-add-labels@v1
+        with:
+          github_token: ${{ secrets.GH_TAGGING_TOKEN }}
+          labels: 'Community'

.github/workflows/no-response.yml

@@ -0,0 +1,20 @@
+name: No Response
+
+on:
+  issue_comment:
+    types: [created]
+  schedule:
+    # Run every day at 9am (UTC time)
+    - cron: '0 9 * * *'
+
+jobs:
+  noResponse:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: lee-dohm/[email protected]
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          responseRequiredLabel: "support: needs more info"
+          daysUntilClose: 28
+          closeComment: >-
+            This issue has been closed due to lack of information. Feel free to re-open this issue if you're facing a similar problem. Please provide as much information as possible so we can help resolve your issue.

ARCHITECTURE.md

@@ -6,6 +6,7 @@ For further information, see also:
 
 - [Kedro-Viz contributing documentation](CONTRIBUTING.md), which covers how to start development on the project
 - [Kedro-Viz style guide](STYLE_GUIDE.md), which walks through our standards and recommended best practices for our codebase
+- [Kedro-Viz Architecture Diagram](https://miro.com/app/board/uXjVKhNg1RE=/?moveToWidget=3458764606468376036&cot=10), to see a high level overview of both back-end and front-end and how they are connected.
 
 ## High-level Overview
 
@@ -62,7 +63,7 @@ The `localStorage` state is updated automatically on every Redux store update, v
 
 ## Data ingestion
 
-![Kedro-Viz data flow diagram](/.github/img/app-architecture-data-flow.png)
+![Kedro-Viz data flow diagram](/.github/img/frontend-architecture.png)
 
 Kedro-Viz currently utilizes two different methods of data ingestion: the Redux setup for the pipeline and flowchart-view related components, and GraphQL via Apollo Client for the experiment tracking components.
 
@@ -147,3 +148,9 @@ Kedro-Viz includes a graph layout engine, for details see the [layout engine doc
 Our layout engine runs inside a web worker, which asynchronously performs these expensive calculations in a separate CPU thread, in order to avoid this blocking other operations on the main thread (e.g. CSS transitions and other state updates).
 
 The app uses [redux-watch](https://github.com/ExodusMovement/redux-watch) with a graph input selector to watch the store for state changes relevant to the graph layout. If the layout needs to change, this listener dispatches an asynchronous action which sends a message to the web worker to instruct it to calculate the new layout. Once the layout worker completes its calculations, it returns a new action to update the store's `state.graph` property with the new layout. Updates to the graph input state during worker calculations will interrupt the worker and cause it to start over from scratch.
+
+## Backend Architecture
+
+![Kedro-Viz backend architecture](/.github/img/backend-architecture.png)
+
+The backend of Kedro-Viz serves as the data provider and API layer that interacts with Kedro projects and manages data access for visualisations in the frontend.  It offers both REST and GraphQL APIs to support data retrieval for the frontend, allowing access to pipeline structures, node-specific details, and experiment tracking data. Key components include the `DataAccessManager`, which interfaces with data `Repositories` to fetch and structure data. The CLI enables users launch with Kedro-Viz from the command line, while deploy and build options enables seamless sharing of pipeline visualisations on any static website hosting platform.

README.npm.md

@@ -140,6 +140,9 @@ The example below demonstrates how to configure your kedro-viz using different `
       tag: {
         enabled: {companies: true}
       },
+      behaviour: { 
+        reFocus: true,
+      },
       theme: "dark"
     }}
 />
@@ -161,6 +164,9 @@ The example below demonstrates how to configure your kedro-viz using different `
 | `sidebar`                  | boolean                                                                            | true                                                     | Show/Hide Sidebar and action toolbar                                                                                                                          |
 | `zoomToolbar`              | boolean                                                                            | true                                                     | Show/Hide zoom-in, zoom-out and zoom reset buttons together                                                                                                   |
 | options.expandAllPipelines | boolean                                                                            | false                                                    | Expand/Collapse Modular pipelines on first load                                                                                                               |
+| options.behaviour            |                                                                                    |                                                          |                                                                                                                                                               |
+| `reFocus`       | boolean                                                                            | true                                                     | In the flowchart, enable or disable the node re-focus behavior when clicking on nodes.
+
 | options.nodeType           | `{disabled: {parameters: boolean,task: boolean,data: boolean}}`                    | `{disabled: {parameters: true,task: false,data: false}}` | Configuration for node type options                                                                                                                           |
 | options.tag                | `{enabled: {<tagName>: boolean}}`                                                  | -                                                        | Configuration for tag options                                                                                                                                 |
 | options.theme              | string                                                                             | dark                                                     | select `Kedro-Viz` theme : dark/light                                                                                                                         |

RELEASE.md

@@ -11,14 +11,19 @@ Please follow the established format:
 ## Major features and improvements
 
 - Update Kedro-Viz telemetry for opt-out model (#2022)
+- Introduce `behaviour` prop object with `reFocus` prop (#2161)
 
 ## Bug fixes and other changes
 
 - Improve `kedro viz build` usage documentation (#2126)
 - Fix unserializable parameters value (#2122)
+- Replace `watchgod` library with `watchfiles` and improve autoreload file watching filter (#2134)
 - Display full dataset type with library prefix in metadata panel (#2136)
 - Enable SQLite WAL mode for Azure ML to fix database locking issues (#2131)
 - Replace `flake8`, `isort`, `pylint` and `black` by `ruff` (#2149)
+- Refactor `DatasetStatsHook` to avoid showing error when dataset doesn't have file size info (#2174)
+- Fix 404 error when accessing the experiment tracking page on the demo site (#2179)
+- Add check for port availability before starting Kedro Viz to prevent unintended browser redirects when the port is already in use (#2176)
 
 
 # Release 10.0.0

docs/source/kedro-viz_visualisation.md

@@ -194,23 +194,64 @@ The visualisation now includes the layers:
 
 ![](./images/pipeline_visualisation_with_layers.png)
 
+Duplicated definitions like:
+
+```yaml
+metadata:
+  kedro-viz:
+    layer: raw
+```
+
+can be avoided by leveraging YAML native syntax for anchors and aliases.
+
+Use an anchor (`&`) first, to create a reusable piece of configuration:
+
+```yaml
+_raw_layer: &raw_layer
+  metadata:
+    kedro-viz:
+      layer: 01_raw
+```
+
+And then use aliases (`*`) to reference it:
+
+```yaml
+companies:
+  type: pandas.CSVDataset
+  filepath: data/01_raw/companies.csv
+  <<: *raw_layer
+
+reviews:
+  type: pandas.CSVDataset
+  filepath: data/01_raw/reviews.csv
+  <<: *raw_layer
+
+# Same for other datasets of the raw layer...
+```
+
+See [this example from the Kedro docs](https://docs.kedro.org/en/stable/data/data_catalog_yaml_examples.html#load-multiple-datasets-with-similar-configuration-using-yaml-anchors) for more details.
+
 ## Share a pipeline visualisation
 
-You can share a pipeline structure within a Kedro-Viz visualisation as a JSON file from the terminal:
+You can save a pipeline structure within a Kedro-Viz visualisation directly from the terminal as follows:
 
 ```bash
 kedro viz run --save-file=my_shareable_pipeline
 ```
 
-This command will save a visualisation of the `__default__` pipeline as a JSON file called `my_shareable_pipeline.json`. It doesn't share data, such as that in the code panel, nor can you share images or charts.
+This command saves your visualisation in a `my_shareable_pipeline` folder, which contains all pipeline and node information from your Kedro project.
 
-To visualise the shared file, type the following to load it from the terminal:
+To visualise your saved Kedro-Viz, load the `my_shareable_pipeline` folder from the terminal with:
 
 ```bash
 kedro viz run --load-file=my_shareable_pipeline
 ```
 
-You can also share a complete project visualisation, described in more detail on [the following page](./share_kedro_viz). 
+```{note}
+This way of sharing requires a Kedro environment setup. 
+
+For users who prefer not to set up a Kedro environment, [Kedro-Viz visualisations can also be shared via multiple hosting solutions](./share_kedro_viz).
+```
 
 ## Running Kedro-viz in a notebook. 
 

package.json

@@ -13,7 +13,7 @@
   },
   "proxy": "http://localhost:4142/",
   "scripts": {
-    "build": "cross-env GENERATE_SOURCEMAP=false react-scripts build",
+    "build": "cross-env GENERATE_SOURCEMAP=false react-scripts build && cp ./build/index.html ./build/404.html",
     "postbuild": "rm -rf build/api",
     "start": "REACT_APP_DATA_SOURCE=$DATA NODE_OPTIONS=\"--dns-result-order=ipv4first\" npm-run-all -p start:app start:lib",
     "start:dev": "rm -rf node_modules/.cache && npm start",

package/features/steps/lower_requirements.txt

@@ -3,7 +3,7 @@ fastapi==0.100.0
 fsspec==2021.4
 aiofiles==22.1.0
 uvicorn[standard]==0.22.0
-watchgod==0.8.2
+watchfiles==0.24.0
 plotly==4.8
 packaging==23.0
 pandas==1.3; python_version < '3.10'
@@ -16,3 +16,4 @@ secure==0.3.0
 # numpy 2.0 breaks with old versions of pandas and this
 # could be removed when the lowest version supported is updated
 numpy==1.26.4
+pathspec==0.12.1

package/kedro_viz/api/apps.py

@@ -15,7 +15,7 @@
 from jinja2 import Environment, FileSystemLoader
 
 from kedro_viz import __version__
-from kedro_viz.api.rest.responses import EnhancedORJSONResponse
+from kedro_viz.api.rest.responses.utils import EnhancedORJSONResponse
 from kedro_viz.integrations.kedro import telemetry as kedro_telemetry
 
 from .graphql.router import router as graphql_router