Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GEN-1076] Exclude genomic_location_explanation from release #548

Merged
merged 6 commits into from
Jan 25, 2024

Conversation

rxu17
Copy link
Contributor

@rxu17 rxu17 commented Jan 24, 2024

Purpose: This is a draft PR. This PR is only allowing in certain release variables (excluding out the genomic_location_explanation and Annotation_Status variables for example) for the full maf csv.

Changes:

  • Added create_missing_columns in process_functions.py Decided with this implementation on excluding variables because we have older full maf files that may have been processed 1-N releases ago, and doesn't contain new columns and features we added to maf processing since then. Because we append old full maf files and more recently processed maf files, we want to make sure they conform to a specific expected schema so we set the expected columns they should have and if they don't have the column, it's created and filled with missing values.

Testing:

  • Unit tests written for the new function
  • Ran on consortium and public release aspects of pipeline
  • Ran in docker image

Depends on #545

genie/database_to_staging.py Outdated Show resolved Hide resolved
@rxu17
Copy link
Contributor Author

rxu17 commented Jan 24, 2024

Also feels like we need to also enforce an expected schema for the full maf file during the end of process_mutation given that we have this new variable genomic_location_explanation.

Copy link

dpulls bot commented Jan 24, 2024

🎉 All dependencies have been resolved !

Base automatically changed from gen-1027-store-error-report to develop January 24, 2024 23:44
@rxu17 rxu17 marked this pull request as ready for review January 25, 2024 00:18
@rxu17 rxu17 requested a review from a team as a code owner January 25, 2024 00:18
Copy link

Quality Gate Passed Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

9 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@rxu17
Copy link
Contributor Author

rxu17 commented Jan 25, 2024

It was decided to just use a list of release variables for the full maf file to subset out the non-release variables.

Copy link
Member

@thomasyu888 thomasyu888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 LGTM! Thanks for doing this! I just had one comment, but going to pre-approve!

@rxu17 rxu17 merged commit 889698d into develop Jan 25, 2024
8 checks passed
@rxu17 rxu17 deleted the gen-1076-exclude-new-gn-var branch January 25, 2024 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants