fix: profile events storage relatively to insertion timestamp #144
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amateima
requested review from
mrice32,
nicholaspai,
melisaguevara,
james-a-morris and
daywiss
as code owners
Comment on lines
+130
to
+144
| // Log the time difference for each deposit event for profiling in datadog | ||
| insertedDeposits.forEach((event) => { | ||
| if (event.blockTimestamp === undefined) return; | ||
| const timeDifference = | ||
| event.createdAt.getTime() - event.blockTimestamp.getTime(); | ||
| this.logger.debug({ | ||
| at: "SpokePoolIndexerDataHandler#profileStoreEvents", | ||
| message: "V3FundsDeposited event profile", | ||
| depositId: event.depositId, | ||
| originChainId: event.originChainId, | ||
| timeDifference, | ||
| createdAt: event.createdAt, | ||
| blockTimestamp: event.blockTimestamp, | ||
| }); | ||
| }); |
There was a problem hiding this comment.
OOC would it make sense to spot-check a few of these values? Since this is an O(N) operation, it could add noticeable milliseconds to the process, and it seems worth optimizing to shave down execution time as much as possible
There was a problem hiding this comment.
at some point we might reduce the number of logs, yes. I will introduce an ENV var to disable this while doing large backfills
| @@ -114,7 +114,56 @@ export class SpokePoolIndexerDataHandler implements IndexerDataHandler { | |||
| ); | |||
| await this.updateNewDepositsWithIntegratorId(newInsertedDeposits); | |||
| await this.spokePoolProcessor.process(storedEvents); | |||
| this.profileStoreEvents(storedEvents); | |||
There was a problem hiding this comment.
Could the two async functions above this be called in parallel?
amateima
added a commit
that referenced
this pull request
Jan 7, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For more accurate profiling, we need to compute the time difference relatively to the time the event was inserted in the DB, not the current
now()server timestamp at logging time.In addition, now we profile fill events storing too