Re-purpose SQLiteSessionStore

[noklam]

kedro-org/kedro-viz#1536

SQliteSessionStore actually isn't needed anymore for kedro-viz since 6 months ago. It has been more apparent that there are lots of useful metadata that can be stored for Monitoring/Ops/development purpose.

For example:

• Storing the run history can be useful for reproduce experiment (i.e. the old Journal)
• Performance / bottleneck visualisation etc.

Alternative:

• Do nothing

[astrojuanlu]

With "repurpose" you mean "bring it back"? I see it's mentioned in the docs https://github.com/kedro-org/kedro-viz/blob/df5a18771a7e676a92abf4b413649883497cd605/CONTRIBUTING.md?plain=1#L210 but I can't find it in the codebase anymore @ravi-kumar-pilla

[noklam]

The SQLiteSessionStore was "temporarily" staying in kedro-viz because it's experimental, we had discussion in the past to move that back to kedro but we never did. The reason for that is experiment tracking users are very limited, Session remains a bit unclear etc.

With more visual plugins that we see built for Kedro, it may makes more sense to decouple this a little bit. With Kedro (or one or multiple plugin) focus on collecting the metadata etc during a Kedro run, and another plugin visualising them with a frontend. Maybe that's just the run history + observability + run command + Git hash + diff.

I think Journal was replaced by experiment tracking. The fact that now it's hidden in a sqlite with weird format make it even harder to view compare to a good old JSON