Extend 'preview' functionality to PartitionedDatasets such as Plotly, Pandas Tables (CSV, Excel)

[robguilarr]

Description

I recently asked in the Slack channel whether it would be valuable to enable this feature. Specifically, I would like to be able to save multiple plotly.JSONDataSet objects as values from a dictionary and display them as multiple outputs using kedro-viz.

Context

The context for this request is that I have a node where the number of plots is not fixed, and I would like to have an automated solution to save multiple plots, regardless of the quantity.

Possible Alternatives

As a possible alternative, I suggest adding a button or dynamic parameter in kedro-viz to allow users to select the desired plot, to avoid interference with the visual space in case of many graphics.

[astrojuanlu]

Thanks a lot for this suggestion @robguilarr! Could you add a code snippet of how do you imagine your catalog entry and corresponding node functions to look like?

[antonymilne]

Very relevant: kedro-org/kedro-plugins#529 and my comments on #783 (comment).

@robguilarr how would you feel about doing this using a PartitionedDataSet? Something like this:

# catalog.yml
multiple_plots:
  type: PartitionedDataset
  filepath: data/plots/
  dataset: 
     type: plotly.JSONDataSet

# node code
def make_plots() -> Dict[str, go.Figure]:
    return {"a": plot_a, "b": plot_b, "c": plot_c}

This would produce files data/plots/a.json, data/plots/b.json, data/plots/c.json. In kedro-viz, we would then show one dataset multiple_plots, which when clicked would show all the plots in the metadata panel.

[robguilarr]

Thanks a lot for this suggestion @robguilarr! Could you add a code snippet of how do you imagine your catalog entry and corresponding node functions to look like?

My situation aligns with what @AntonyMilneQB previously mentioned. Specifically, the primary focus of interest would be centered around the presentation within kedro-viz + "How you save it in a single catalog definition". As an illustration, I would like to highlight that I don't have a predetermined quantity of plots, which consequently would affect the visual representation of my node output:

{"plot_AvB": plotly.JSONDataSet, "plot_AvC": plotly.JSONDataSet, ..... , "plot_ZvZ": plotly.JSONDataSet}

And then be saved as mentioned by Antony, I agree with that catalog definition

[antonymilne]

This makes a lot of sense to I think. Unfortunately it's not super easy to do and I'm not sure how much demand there is for it, so not sure how high priority it will be. But personally it's something I would like to see added too 👍

[tynandebold]

@antonymilne we're discussing this now in backlog grooming and would love if you could sketch this out:

which when clicked would show all the plots in the metadata panel.

How do you see that looking in Viz?

[antonymilne]

Not sure to be honest... Maybe like a big long list of pictures that makes the metadata panel scrollable? Or little thumbnails which you can then expand in a new modal? Like it is at the moment but the thumbnails would need to be way smaller to fit many more on the page. Or even simpler, just some text that says "multiple plots here, click this button to scroll through them" which thens opens the first one in the modal and you can then browse others by going previous/next on that modal. Or something similar to how we show many plots on the experiment tracking screen.

Ultimately there's many ways to do it and I don't have much feeling about which would be best from a balance of UX and engineering effort. One important requirement is that the solution works for any number of plots though (the number is known in advance), not just a couple.

[robguilarr]

From my perspective as a user, the option multiple plots here, click this button to scroll through them sounds like a more reasonable way, in the sense of not cluttering up the UI :)