Update exercise for 2024 with feedback form 2023

[afoix]

This pull request includes:

• Update in the README.md including the format suggested in this issue . This solve issue Updated the readme.md following the suggestions  #12 .
• Add feedback suggested in 2023: work together with the KE task. This solves issue Improvement suggestions from 2023 #8 .
• Add feedback suggested in 2023: Add an exercises combining the two dataset sets at the end. This solves issue Improvement suggestions from 2023 #8
• We tested to include Colored MNIST but I did not manage to find a narrative that flows so we kept the standard MNIST.

[cmalinmayor]

@afoix How long did it take to train the denoising model for you? On my M3 mac (no GPU, but usually quite speedy with pytorch) its over half an hour. We should test on the actual hardware, because if it is that slow we need to make it faster.

[cmalinmayor]

I still want to test the denoising part on the hardware to see how long it takes. I don't remember it being slow last year? Perhaps @adjavon has more memory on this than I do. @adjavon also curious to get your opinion on the new denoising tasks - at the very least we should explain them a bit more.

[cmalinmayor]

I think we need to think a bit more on what the point of this part is. Do we actually want them to train sequentially and then shuffled? I think it would be fine to just train shuffled and show that it works and is the correct way to get good (ish?) performance on both. If we want to include a sequential example, I think it would make more sense as essentially a fine-tuning procedure, where you fully train on MNIST and then FashionMNIST, which would show how fine-tuned networks forget their initial task. The way it is implemented now, both datasets are included in both epochs so it is not really sequential or shuffled, and I find it confusing what exactly we are trying to teach them. Also, if the denoiser takes longer than 10 minutes to train, we should definitely skip sequential.

[afoix]

Hey @cmalinmayor the denoiser definitely takes less to train at least in my machine with a GPU 😄 However, I am happy to skip the sequential part if you both prefer it to be removed. Let me know! :)

[adjavon]

I added a comment below, but long story short on the AWS hardware it is just fast enough to not be too annoying. I like having both because I like the lesson I learn from it (networks forget). I can have a go at making that a bit more on-the-nose in the text during my afternoon run typos-and-things PR :)

[afoix]

@afoix How long did it take to train the denoising model for you? On my M3 mac (no GPU, but usually quite speedy with pytorch) its over half an hour. We should test on the actual hardware, because if it is that slow we need to make it faster.

Hello @cmalinmayor in my machine, using a GPU each epoch is only 25 seconds, and it's 5 epochs so it's actually quite fast 😄

[adjavon]

I still want to test the denoising part on the hardware to see how long it takes. I don't remember it being slow last year? Perhaps @adjavon has more memory on this than I do. @adjavon also curious to get your opinion on the new denoising tasks - at the very least we should explain them a bit more.

I'm testing the whole exercise today on the TA machine so we can see how long it takes.

[adjavon]

Notes while running the exercise on the TA machine

1. Something seems wrong with the normalization of the confusion matrices when I run this; it taps out at 30 and therefore the colors don't really match the numbers. For instance in the example below the cells on row 4 that have values 45, 17, and 34 are colored the same way as the diagonal cells that have values >90.

2. Generally, the TQDM progress bars disappear after each epoch is done and turn into an odd recap (see first line). Not sure if this is on purpose as I've never seen this before, but ignore if it is!

3. The first denoising network takes about 40s per epoch to train on the TA machine -- just the right amount of time for me to go make myself a coffee -- perfectly reasonable :)
4. The second denoising network (on the concatenated dataset) takes 1min30 seconds per epoch on the TA machine; falls into the longer side but still generally reasonable.
5. Same for the third denoising network (shuffled combined). I am not against keeping it as it does provide an important learning point (i.e. networks forget)
6. A suggestion: it would be nice to have a little "Conclusion" at the final checkpoint about key takeaways

I noticed a couple typos and trivial things that I can fix so I'll make a PR this afternoon for those :)

[cmalinmayor]

Something seems wrong with the normalization of the confusion matrices when I run this; it taps out at 30 and therefore the colors don't really match the numbers. For instance in the example below the cells on row 4 that have values 45, 17, and 34 are colored the same way as the diagonal cells that have values >90.

It seems like it is coloring on absolute numbers, not on percents. Weird!! We should fix.

[adjavon]

Something seems wrong with the normalization of the confusion matrices when I run this; it taps out at 30 and therefore the colors don't really match the numbers. For instance in the example below the cells on row 4 that have values 45, 17, and 34 are colored the same way as the diagonal cells that have values >90.

It seems like it is coloring on absolute numbers, not on percents. Weird!! We should fix.

Will try to fix in my PR Done

[afoix]

@adjavon @cmalinmayor thank you for your help! I am merging this now onto main ready for the course 🥳