enhance: [2.4]Determine the number of buffers based on the resource limits of the DataNode

[xiaocai2333]

issue: #28410

master pr: #38209

[mergify]

@xiaocai2333

Invalid PR Title Format Detected

Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:

1. Title Format: The PR title must begin with one of these prefixes:

• feat: for introducing a new feature.
• fix: for bug fixes.
• enhance: for improvements to existing functionality.
• test: for add tests to existing functionality.
• doc: for modifying documentation.
• auto: for the pull request from bot.

2. Description Requirement: The PR must include a non-empty description, detailing the changes and their impact.

Required Title Structure:

[Type]: [Description of the PR]

Where Type is one of feat, fix, enhance, test or doc.

Example:

enhance: improve search performance significantly 

Please review and update your PR to comply with these guidelines.

[codecov]

Codecov Report

Attention: Patch coverage is 31.25000% with 55 lines in your changes missing coverage. Please review.

Project coverage is 72.15%. Comparing base (a516475) to head (5e1a561).
Report is 6 commits behind head on 2.4.

Files with missing lines	Patch %	Lines
...ternal/datanode/compaction/clustering_compactor.go	30.37%	55 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              2.4   #38210      +/-   ##
==========================================
- Coverage   80.40%   72.15%   -8.26%     
==========================================
  Files        1074     1074              
  Lines      168650   168698      +48     
==========================================
- Hits       135609   121719   -13890     
- Misses      28589    42544   +13955     
+ Partials     4452     4435      -17     

Files with missing lines	Coverage Δ
pkg/util/paramtable/component_param.go	98.36% <100.00%> (ø)
...ternal/datanode/compaction/clustering_compactor.go	64.78% <30.37%> (-1.79%)	⬇️

... and 237 files with indirect coverage changes

[czs007]

/approve
/lgtm

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: czs007, xiaocai2333

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• configs/OWNERS [czs007]
• internal/datanode/OWNERS [czs007]
• pkg/util/OWNERS [czs007]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment