Add documentation for versioning with DVC in Kedro

[lrcouto]

Description

Development notes

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

• Read the contributing guidelines
• Signed off each commit with a Developer Certificate of Origin (DCO)
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added a description of this change in the RELEASE.md file
• Added tests to cover my changes
• Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

[ankatiyar]

Overall looks good so far! Would be nice to also add outputs to some of these steps.

[ankatiyar]

I didn't explore the VS Code extension myself but would it work with a Kedro project? If so, would be nice to include a section on how to use it.

[lrcouto]

I can give it a try!

[astrojuanlu]

Rendered version https://kedro--4443.org.readthedocs.build/en/4443/data/kedro_dvc_versioning.html

[astrojuanlu]

Tried to follow the instructions but found some blockers, so I didn't review all of the document

[astrojuanlu]

There's nothing to suppose, the spaceflights-pandas has this dataset right? Maybe reword it as "Verify that your conf/base/catalog.yml contains this dataset"

[astrojuanlu]

Although really this is about data/01_raw/companies.csv being present 🤔

[astrojuanlu]

What about

Suggested change

	Suppose you have a dataset in your project, such as:
	Verify that your project catalog contains this dataset definition:

[astrojuanlu]

Sorry for being a pain in the neck 🙏🏼

I think we need to decide who's the audience of this guide.

• Is it users that don't know much about Kedro or DVC?
• Or is it Kedro users that already know their way around more or less?

In the first case, I think this should almost read like a tutorial and the commands and the steps should be fool-proof. For example, instead of clarifying about the .gitignore after dvc add, we should add a step before that reads like

Since the spaceflights-pandas starter ignores everything under data/ by default, you have to update the .gitignore file as follows:

Otherwise, if we assume that the reader knows a bit about Kedro, we can make the guide less prescriptive. In that case, then the sentence from the beginning "For this example, we will be using a Kedro spaceflights-pandas starter project" should be modified, maybe like

For example, you can use the spaceflights-pandas starter, see the documentation about starters [link]

---

As an example of this, I've been always inspired by the work @/melissawm did on the NumPy tutorials, for example

• https://numpy.org/numpy-tutorials/content/tutorial-svd.html
• https://numpy.org/numpy-tutorials/content/tutorial-ma.html

We don't have to follow the same structure but at least having a clear reader persona in mind helps structure the rest of the document 💡

[lrcouto]

Sorry for being a pain in the neck 🙏🏼

I think we need to decide who's the audience of this guide.

* Is it users that don't know much about Kedro or DVC?

* Or is it Kedro users that already know their way around more or less?

In the first case, I think this should almost read like a tutorial and the commands and the steps should be fool-proof. For example, instead of clarifying about the .gitignore after dvc add, we should add a step before that reads like

Since the spaceflights-pandas starter ignores everything under data/ by default, you have to update the .gitignore file as follows:

Otherwise, if we assume that the reader knows a bit about Kedro, we can make the guide less prescriptive. In that case, then the sentence from the beginning "For this example, we will be using a Kedro spaceflights-pandas starter project" should be modified, maybe like

For example, you can use the spaceflights-pandas starter, see the documentation about starters [link]

As an example of this, I've been always inspired by the work @/melissawm did on the NumPy tutorials, for example

* https://numpy.org/numpy-tutorials/content/tutorial-svd.html

* https://numpy.org/numpy-tutorials/content/tutorial-ma.html

We don't have to follow the same structure but at least having a clear reader persona in mind helps structure the rest of the document 💡

These are some fair points. I think the step-by-step tutorial approach might be better, I usually don't like to make assumptions about an user's knowledge level.

[astrojuanlu]

Still some more work needed

[astrojuanlu]

This git commit comes at a weird moment because we haven't added any other file yet.

For example, we could tell the user to do git init followed by git add . && git commit -m 'First commit, initial structure from starter'.

[astrojuanlu]

Also important to tell the user that the companies.csv file will "disappear"

[astrojuanlu]

Oh and a data/01_raw/.gitignore file is added!

[astrojuanlu]

This is how the .gitignore looked like initially, I had to comment out this line

[astrojuanlu]

s3://mybucket will not exist, unless we tell the user to create it.

We can either use MinIO and add the required steps, or move this section towards the bottom as an "example" of something the user could do (in other words: keep the first half of this document as a step-by-step tutorial, and the second half more like a how-to guide https://diataxis.fr/)

[astrojuanlu]

Suggested change

	- Artefacts and metrics cannot be tracked effectively.
	- Artifacts and metrics cannot be tracked effectively.

[astrojuanlu]

Also what does this mean in practice?