ADBDEV-6156 Count startup memory of each process when using resource groups #1023
Conversation
|
Failed job Deploy multiarch Dockerimages: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1807317 |
|
Allure report https://allure.adsw.io/launch/78383 |
|
Failed job Resource group isolation tests on x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1807327 |
|
Failed job Resource group isolation tests on ppc64le: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1807328 |
|
Allure report https://allure.adsw.io/launch/78465 |
|
Failed job Regression tests with ORCA on x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1817041 |
|
Failed job Regression tests with Postgres on x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1817039 |
|
Failed job Resource group isolation tests on ppc64le: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1817048 |
|
Failed job Resource group isolation tests on x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1818658 |
|
Failed job Resource group isolation tests on ppc64le: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1818659 |
|
Can you write some tests to check? |
|
Failed job Build for x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1850837 |
| DROP | ||
|
|
||
| -- start_ignore | ||
| ! gpstop -rai; |
There was a problem hiding this comment.
may be add
! gpconfig -r gp_resource_manager;before this line?
There was a problem hiding this comment.
This is done in disable_resgroup test
|
|
|
Failed job Build for x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1880757 |
|
Failed job Build for x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1890216 |
|
Allure report https://allure.adsw.io/launch/79897 |
|
Failed job Resource group isolation tests on x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1891649 |
|
Allure report https://allure.adsw.io/launch/80245 |
|
Failed job Resource group isolation tests on x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1911167 |
|
Failed job Build ubuntu22 for x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1924163 |
|
Failed job Build for x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1924162 |
|
Allure report https://allure.adsw.io/launch/80289 |
|
Failed job Resource group isolation tests on x86_64: https://gitlab.adsw.io/arenadata/github_mirroring/gpdb/-/jobs/1926814 |
|
Allure report https://allure.adsw.io/launch/80301 |
|
Allure report https://allure.adsw.io/launch/90926 |
|
"but after the backend is assigned a resource group, this memory is not counted as consumed by the group." - it has been fixed, so let's write in past tense |
src/test/isolation2/input/resgroup/resgroup_startup_memory.source
Outdated
Show resolved
Hide resolved
src/test/isolation2/input/resgroup/resgroup_startup_memory.source
Outdated
Show resolved
Hide resolved
|
Allure report https://allure.adsw.io/launch/91028 |
|
Allure report https://allure.adsw.io/launch/91031 |
|
Allure report https://allure.adsw.io/launch/91084 |
|
Allure report https://allure.adsw.io/launch/92112 |
src/test/isolation2/input/resgroup/resgroup_startup_memory.source
Outdated
Show resolved
Hide resolved
src/test/isolation2/input/resgroup/resgroup_startup_memory.source
Outdated
Show resolved
Hide resolved
src/test/isolation2/input/resgroup/resgroup_startup_memory.source
Outdated
Show resolved
Hide resolved
src/test/isolation2/input/resgroup/resgroup_startup_memory.source
Outdated
Show resolved
Hide resolved
src/test/isolation2/input/resgroup/resgroup_startup_memory.source
Outdated
Show resolved
Hide resolved
src/test/isolation2/input/resgroup/resgroup_startup_memory.source
Outdated
Show resolved
Hide resolved
|
Allure report https://allure.adsw.io/launch/92267 |
|
Allure report https://allure.adsw.io/launch/92334 |
Count the startup memory of each active process when using resource groups
Make the resource manager track the startup memory of each active backend so
that the runaway detector would estimate memory more accurately.
The startup memory is the memory that the backend consumes after startup before
the memory managers (Vmem tracker and resource groups) are initialized. The Vmem
tracker counts this memory as consumed by the segment, but after the backend was
assigned a resource group, this memory was not counted as consumed by the group.
This patch adds startup memory consumption to self->memUsage to make resource
groups consider this memory.
Additionally, this patch slightly modifies the resGroupPalloc function so that
it takes startup memory into account. This is necessary to avoid changing or
complicating the logic of existing tests.
It is worth noting that this patch fixes the accounting of memory consumption by
only active backends (which execute the query). Accounting for the memory
occupied by idle backends is a more complex task that should be done separately.