Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GEN-868] add year death validation #560

Merged
merged 9 commits into from
Apr 24, 2024

Conversation

danlu1
Copy link
Contributor

@danlu1 danlu1 commented Apr 22, 2024

Problem:

Lack of validation check on whether YEAR_DEATH >= YEAR_CONTACT

Solution:

Add the validation check to compare YEAR_DEATH and YEAR_CONTACT when both of them exist and not NAs.

Testing:
Unit test has been added.

@danlu1 danlu1 requested a review from a team as a code owner April 22, 2024 18:52
@danlu1 danlu1 marked this pull request as draft April 22, 2024 18:52
Copy link
Contributor

@rxu17 rxu17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this! Just some initial comments.

Edit: Could you add more details to the PR description?

genie_registry/clinical.py Outdated Show resolved Hide resolved
genie/load.py Outdated Show resolved Hide resolved
genie_registry/clinical.py Outdated Show resolved Hide resolved
genie_registry/clinical.py Outdated Show resolved Hide resolved
@danlu1
Copy link
Contributor Author

danlu1 commented Apr 22, 2024

Thanks for doing this! Just some initial comments.

Edit: Could you add more details to the PR description?

Will do

genie_registry/clinical.py Outdated Show resolved Hide resolved
genie_registry/clinical.py Outdated Show resolved Hide resolved
@danlu1 danlu1 closed this Apr 22, 2024
@danlu1 danlu1 deleted the gen-868-add-year-death-validation branch April 22, 2024 22:42
@danlu1 danlu1 restored the gen-868-add-year-death-validation branch April 22, 2024 22:42
@danlu1 danlu1 reopened this Apr 22, 2024
@danlu1 danlu1 force-pushed the gen-868-add-year-death-validation branch from 9e41546 to 41b0356 Compare April 23, 2024 02:15
@danlu1 danlu1 marked this pull request as ready for review April 23, 2024 02:19
if haveColumn:
error = _check_year_death_validity(clinicaldf=clinicaldf)
total_error.write(error)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A stretch goal outside of the explicit scope of this change would be to split out these other items into their own functions. Is there a tech debt jira that already exists, or could one be created to address this item?

Cc: @rxu17

Copy link
Contributor

@rxu17 rxu17 Apr 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we have a tech debt jira/part of the overarching goal for this. The goal is to eventually have a validation function for each validation rule for each fileformat especially when we want to implement GX framework for this.

I think this is part of our documentation epic as well. To break up the validation code so we can document each validation rule individually and completely.

This is not just happening here but throughout the genie code (also for processing as well) but we definitely want to start with validation first.

Copy link

@BryanFauble BryanFauble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work! Just a few formatting and follow up items I noted.

@danlu1 danlu1 requested a review from rxu17 April 23, 2024 17:06
genie_registry/clinical.py Outdated Show resolved Hide resolved
genie_registry/clinical.py Outdated Show resolved Hide resolved
Copy link
Member

@thomasyu888 thomasyu888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 I'm going to pre-approve! great work here!

genie_registry/clinical.py Outdated Show resolved Hide resolved
@rxu17 rxu17 self-requested a review April 24, 2024 21:37
Copy link
Contributor

@rxu17 rxu17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also going to pre-approve! (just have the row-based validation implementation and addressing some of the comments)

Copy link

Quality Gate Passed Quality Gate passed

Issues
20 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@danlu1 danlu1 merged commit 2985f37 into develop Apr 24, 2024
14 checks passed
@rxu17 rxu17 deleted the gen-868-add-year-death-validation branch April 26, 2024 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants