Compact legendre polynomials

[lukasm91]

Legendre polynomials don't need to be stored zero-padded, we can just concatenate them with proper zero padding.

E.g. on tco2559, this saves up to almost 40GB per rank (used to be 64 GB, now it is 26 GB). Initially, we did this for Leonardo, because legendre coefficients used almost the whole device memory.

[lukasm91]

(convert to draft for the moment ==> it is fnished but we should first merge the other PR, then rebase and review)

[lukasm91]

Beside replacing ext_acc with the proper "copy module", we are only left with this PR from the old GPU branch which I now rebased on top of develop. This PR is compacting the legendre polynomials and with this removing zero padding.

This PR is not expected to interfere with any other PR, as it only touches the GEMMs. Since I am touching the CUDA interfaces anyway, I also added const to pointers in the interface.

• Applied clang-format
• Tested that everything is properly deallocated
• Tested large sizes for no overflows.

@samhatfield @wdeconinck Feel free to review when you find time.

[wdeconinck]

I think this looks all fine and what a nice saving!
Perhaps @samhatfield can run it in few places to double check all is in order ?

[samhatfield]

If I can find the time this week, I will take a look. It might have to wait until next year.

[samhatfield]

Sorry, I am gradually getting through this finally :) Will add some more comments soon.

[samhatfield]

Sorry @lukasm91, #180 has introduced a merge conflict, but it looks quite simple to resolve.

[lukasm91]

no worry - I merged it with develop.

[samhatfield]

If I understand right, you have basically flattened the Legendre polynomial work arrays (ZAA, ZAS), which contain the polynomials for every zonal wavenumber, from 3D to 1D, borrowing the same offset indexing style of the other 1D work arrays. The 3D representation was wasteful because we allocated the same amount of memory for each zonal wavenumber despite that the size of the polynomial array decreases as you go up in wavenumber. Very nice work indeed.

I am happy to merge this pending the conversation above and once you give me the go ahead @lukasm91.

[lukasm91]

I just modified the consts to be "consistent" now. Your description makes sense! I think from my side we are ready to merge.