[Enhancement] Avoid acquire multiple lock in the same stack for tablet report #55808
+46
−20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…t report ## Why I'm doing: In current impl, the longest lock acquire chain in tablet report will be looks like: tablet shard lock -> tablet meta lock -> pk relative lock which may make other thread stuck easliy because too many lock acquire at the same time. ## What I'm doing: Remove the lock dependency. Signed-off-by: srlch <[email protected]>
![starrocks-cr[bot]](https://avatars.githubusercontent.com/in/404334?s=60&v=4){kind=small}
| } | ||
| } | ||
| all_tablets.clear(); | ||
|
|
||
| LOG(INFO) << "Report all " << tablets_info->size() | ||
| << " tablets info. max_tablet_rowset_num:" << max_tablet_rowset_num; | ||
| StarRocksMetrics::instance()->max_tablet_rowset_num.set_value(max_tablet_rowset_num); |
There was a problem hiding this comment.
The most risky bug in this code is:
Potential deadlock when accessing _tablets_shards.
You can modify the code like this:
std::vector<TabletSharedPtr> all_tablets;
for (const auto& tablets_shard : _tablets_shards) {
// Ensure lock acquisition is done inside a smaller scope to prevent prolonged holding of locks
{
std::shared_lock rlock(tablets_shard.lock);
for (const auto& [_, tablet_ptr] : tablets_shard.tablet_map) {
all_tablets.push_back(tablet_ptr);
}
}
}
size_t max_tablet_rowset_num = 0;
for (auto& tablet_ptr : all_tablets) {
TTablet t_tablet;
TTabletInfo tablet_info;
tablet_ptr->build_tablet_report_info(&tablet_info);
max_tablet_rowset_num = std::max(max_tablet_rowset_num, tablet_ptr->version_count());
// find expired transaction corresponding to this tablet
TabletInfo tinfo(tablet_ptr->tablet_id(), tablet_ptr->schema_hash(), tablet_ptr->tablet_uid());
auto find = expire_txn_map.find(tinfo);
if (find != expire_txn_map.end()) {
tablet_info.__set_transaction_ids(find->second);
expire_txn_map.erase(find);
}
t_tablet.tablet_infos.push_back(tablet_info);
if (!t_tablet.tablet_infos.empty()) {
tablets_info->emplace(tablet_ptr->tablet_id(), t_tablet);
}
}
all_tablets.clear();
srlch
changed the title
Signed-off-by: srlch <[email protected]>
[FE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
[Java-Extensions Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
[BE Incremental Coverage Report]✅ pass : 20 / 22 (90.91%) file detail
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I'm doing:
In current impl, the longest lock acquire chain in tablet report will be looks like: tablet shard lock -> tablet meta lock -> pk relative lock which may make other thread stuck easliy because too many lock acquire at the same time.
What I'm doing:
Remove the lock dependency.
Fixes #issue
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: