Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: use JobProgress tracking in data value import [DHIS2-18885] #19781

Merged
merged 10 commits into from
Feb 5, 2025

Conversation

jbee
Copy link
Contributor

@jbee jbee commented Jan 27, 2025

Summary

This started as a cleanup PR for Notifier stuff but I found that the ADX import multi-threading solution messed up the notification messages as messages from different threads were writing into the same notification list. Looking into that I ended up removing the cause of the problem by streamlining the ADX data value import.

  • makes ADX fully synchronous (no user context issues, no notification mixup)
  • adds progress tracking in the data value set import to get more insight into the import
  • adds JobProgress#setWorkItemBucketing to prevent too many items by "merging" n items into one "bucket" item

Automatic Testing

Existing tests were adjusted.

Manual Testing

  • goto import/export app
  • export some data as ADX
  • import the ADX file again
  • also try importing other formats, JSON, XML and observe the job progress shown

@jbee jbee self-assigned this Jan 27, 2025
@jbee jbee changed the title chore: use JobProgress tracking in data value import [DHIS2-18885] refactor: use JobProgress tracking in data value import [DHIS2-18885] Jan 27, 2025
@jbee jbee marked this pull request as ready for review February 4, 2025 16:14
Copy link

sonarqubecloud bot commented Feb 5, 2025

@jbee jbee merged commit 968c1d2 into master Feb 5, 2025
17 checks passed
@jbee jbee deleted the DHIS2-18885-1 branch February 5, 2025 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants