The maximum grid size?

[WuJunxu]

OrdinaryKriging: I run with 100 samples and 1000x1000 grid size, got MemoryError, but succeed with 500x500, so how large grid size can be supported?

[MuellerSeb]

Hey there,

it is depending on the memory size of your system. For example, I have 16GB of memory installed and I don't have a problem with a grid-size of 1000*1000. The kriging matrix has a size of at least n_samples * n_samples and the right-hand-side of n_samples * n_grid_pnts.

To get an impression, have a look at the following code:

>>> import numpy as np
>>> a = np.zeros((32768, 32768), dtype=float) 
>>> a.nbytes/1024/1024
8192.0

So a numpy array of the size 2^15 * 2^15 takes 8GiB of RAM.

We are working on more memory efficient routines, but for now, you have to either down-size your input-data, or update your system-memory.

If you are on a Linux system, you could think about using a dynamic swap-space to increase available ram by using your HDD/SSD (helped me some years ago): https://github.com/Tookmund/Swapspace

Cheers,
Sebastian

[markbroich]

Hello Sebastian,

tnx for your prior msg.

Are 1k^2 points w 16 gb ram still the ~limit? Does it matter if style = ‘grid’ or ‘points’?
Also, does krig in R have similar constrains re the max number of points?
I generally prefer python but I may need to interpolate some rather large datasets.
Just curious.

Tnx for your work. Bis bald, Mark

[MuellerSeb]

As I wrote, the limit should be around 40000 points. grid will just simplify the points definition. A grid with 100100 axis points will result in 10000 points. There is no difference between passing the 100100 grid and passing the 10000 points directly.

I don't know about the implementation details in R, you should look up their respective documentation.

Cheers, Sebastian