Stuck with the creation of tidy_pmi

[marcelbaumgartner]

Just discovered this wonderful book. Amazing job, congratulations! In Chapter 5, I try to create the tidy_pmi tibble, using the slide_windows function you created. Using a Windows machine with 16GB memory. But I never managed to end the calculation. How long does this normally take? Any suggestions to adapt the parallel processing.

sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

[EmilHvitfeldt]

Hello @marcelbaumgartner!! This part of the code does indeed take a long time to run. We extracted out that bit of code to be cached separately than everything else as you can see here https://github.com/EmilHvitfeldt/smltar/blob/master/inst/tidy_pmi.R.

This is definitely a task that takes hours rather than minutes. One thing that might help you is to downsample the data a bit. If you downsample to 1% data you could extrapolate how long the full runtime would be.

[marcelbaumgartner]

Thanks, appreciate your quick response. Clear, will try it on a smaller sample, and will then run it for an entire night :). Best regards from Switzerland, Marcel