diff --git a/README.md b/README.md index 32f80a74..940694a7 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,11 @@ # Medicare CCLF Connector -## 🔗  Docs +## 🔗 Docs Check out our [docs](https://thetuvaproject.com/) to learn about the project and how you can use it.

-## 🧰  What does this repo do? +## 🧰 What does this repo do? The Medicare CCLF Connector is a dbt project that maps raw Medicare CCLF claims data to the Tuva Input Layer, which is the first step in running the Tuva Project. This connector expects your CCLF data to be organized into the tables outlined in this [CMS data dictionary](https://www.cms.gov/files/document/cclf-information-packet.pdf), which is the most recent format CMS uses to distribute CCLF files.

@@ -18,7 +18,7 @@ The Medicare CCLF Connector is a dbt project that maps raw Medicare CCLF claims - Snowflake

-## ✅  Quickstart Guide +## ✅ Quickstart Guide ### Step 1: Clone or Fork this Repository Unlike [the Tuva Project](https://github.com/tuva-health/the_tuva_project), this repo is a dbt project, not a dbt package. Clone or fork this repository to your local machine. @@ -29,7 +29,39 @@ Next you need to import the Tuva Project dbt package into the Medicare CCLF Conn

### Step 3: Data Preparation -The CCLF file specification does not have a field that can be mapped directly to `enrollment_start_date`; therefore, we have added a field called `bene_member_month`. We recommend parsing the monthly enrollment file date from the Beneficiary Demographics filename (e.g., P.A****.ACO.ZC8Y**.Dyymmdd.Thhmmsst) and mapping this date to `bene_member_month`. The connector will handle the rest of the mapping. + +#### Source data: +The source table names the connector is expecting can be found in the +`_sources.yml` config file. You can rename your source tables if needed or add an +alias to the config. + +#### File Dates: +The field `file_date` is used throughout this connector to deduplicate data +received across regular and run-out CCLFs. We recommend parsing this date from +the filename (e.g., P.A****.ACO.ZC1Y**.Dyymmdd.Thhmmsst) and formatting it as +"YYYY-MM-DD". + +#### Enrollment Dates: +The CCLF specification does not have a field that can be mapped directly +to `enrollment_start_date` and `enrollment_end_date`, and the Part A and Part B +entitlement dates (BENE_PART_A_ENRLMT_BGN_DT, BENE_PART_B_ENRLMT_BGN_DT) are +often incorrect or not useful for claims analytics. + +We have included an additional source called `Enrollment` that can be +populated with enrollment dates relevant to your data. These enrollment +dates may come from an attribution file, beneficiary alignment report (BAR), or +any source you may have. You just need to create a source table with the +following columns: + + 1. `current_bene_mbi_id` + 2. `enrollment_start_date` + 3. `enrollment_end_date` + 4. `bene_member_month` + * The connector includes logic to handle enrollment spans or member months. + * If enrollment spans are available, leave this field null. + * If enrollment spans are not available, populate this field with member + month dates in the format "YYYY-MM-DD" and set the variable + `member_months_enrollment` to true in the `dbt_project.yml` file.

### Step 4: Configure Input Database and Schema @@ -37,13 +69,15 @@ Next you need to tell dbt where your Medicare CCLF source data is located. Do t

### Step 5: Run -Now you're ready to run the connector and the Tuva Project. For example, using dbt CLI you would `cd` to the project root folder in the command line and execute `dbt build`. Next you're now ready to do claims data analytics! +Finally, run the connector and the Tuva Project. For example, using dbt CLI you would `cd` to the project root folder in the command line and execute `dbt build`. + +Now you're ready to do claims data analytics!

-## 🙋🏻‍♀️ How do I contribute? +## 🙋🏻‍♀️ How do I contribute? Have an opinion on the mappings? Notice any bugs when installing and running the project? If so, we highly encourage and welcome feedback! While we work on a formal process in Github, we can be easily reached on our Slack community.

-## 🤝 Join our community! +## 🤝 Join our community! Join our growing community of healthcare data practitioners on [Slack](https://join.slack.com/t/thetuvaproject/shared_invite/zt-16iz61187-G522Mc2WGA2mHF57e0il0Q)! diff --git a/dbt_project.yml b/dbt_project.yml index 803d83b9..03f8c417 100644 --- a/dbt_project.yml +++ b/dbt_project.yml @@ -7,11 +7,14 @@ profile: default vars: ## Set the database and schema where raw CCLF data is stored - input_database: dev_ci_testing - input_schema: cclf + input_database: tuva + input_schema: raw_data claims_enabled: true + ## Uncomment this variable if enrollment source is member months +# member_months_enrollment: true + model-paths: ["models"] analysis-paths: ["analyses"] test-paths: ["tests"] @@ -24,12 +27,11 @@ clean-targets: - "target" - "dbt_packages" -# models: -# medicare_cclf_connector: -# +database: default -# the_tuva_project: -# +database: default - -# seeds: -# +database: default - +models: + medicare_cclf_connector: + final: + +materialized: table + intermediate: + +materialized: table + staging: + +materialized: view diff --git a/models/_models.yml b/models/_models.yml index aaf6e7de..a9e8fbf1 100644 --- a/models/_models.yml +++ b/models/_models.yml @@ -9,99 +9,356 @@ models: config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}{{var('tuva_schema_prefix')}}_input_layer{% else %}input_layer{%- endif -%} - materialized: table + tests: + - dbt_utils.unique_combination_of_columns: + combination_of_columns: + - member_id + - enrollment_start_date + - enrollment_end_date + columns: + - name: patient_id + tests: + - not_null + - name: member_id + tests: + - not_null + - name: enrollment_start_date + tests: + - not_null: + config: + severity: warn + - name: enrollment_end_date + tests: + - not_null: + config: + severity: warn - name: medical_claim description: Claim line level medical data. config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}{{var('tuva_schema_prefix')}}_input_layer{% else %}input_layer{%- endif -%} - materialized: table + tests: + - dbt_utils.unique_combination_of_columns: + combination_of_columns: + - claim_id + - claim_line_number + columns: + - name: claim_id + tests: + - not_null + - name: claim_line_number + tests: + - not_null + - name: patient_id + tests: + - not_null + - name: member_id + tests: + - not_null + - name: paid_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: allowed_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: charge_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: coinsurance_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: copayment_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: deductible_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: total_cost_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn - name: pharmacy_claim description: Claim line level pharmacy data. config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}{{var('tuva_schema_prefix')}}_input_layer{% else %}input_layer{%- endif -%} - materialized: table + tests: + - dbt_utils.unique_combination_of_columns: + combination_of_columns: + - claim_id + - claim_line_number + columns: + - name: claim_id + tests: + - not_null + - name: claim_line_number + tests: + - not_null + - name: patient_id + tests: + - not_null + - name: member_id + tests: + - not_null + - name: ndc_code + tests: + - not_null: + config: + severity: warn + - name: paid_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: allowed_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: charge_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: coinsurance_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: copayment_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn + - name: deductible_amount + tests: + - dbt_utils.accepted_range: + min_value: 0 + config: + severity: warn ### intermediate - - name: diagnosis_pivot + - name: int_beneficiary_demographics_deduped + description: Dedupe attribution. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: beneficiary_demographics_deduped + + - name: int_beneficiary_xref_deduped + description: Dedupe beneficiary xref. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: beneficiary_xref_deduped + + - name: int_diagnosis_deduped + description: Dedupe diagnosis codes before pivot. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: diagnosis_deduped + + - name: int_diagnosis_pivot description: > Pivot of diagnosis from long (1 row/diagnosis) to wide (multiple diagnosis per row). config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: diagnosis_pivot + + - name: int_dme_claim_adr + description: Applying adjustment logic for part B DME claims. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: dme_claim_adr - - name: dme_claims + - name: int_dme_claim_deduped description: > - Mapping of durable medical equipment claims to Tuva claims data model. + Final de-duplication, transformation logic, and mapping missing fields. config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: dme_claim_deduped - - name: institutional_claims + - name: int_enrollment description: > - Mapping of institution claims to Tuva claims data model. + Prepare enrollment data, convert member months to spans if needed. config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: enrollment - - name: physician_claims + - name: int_institutional_claim_adr + description: Applying adjustment logic for part A institutional claims. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: institutional_claim_adr + + - name: int_institutional_claim_deduped description: > - Mapping of professional claims to Tuva claims data model. + Final de-duplication, transformation logic, and mapping missing fields. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: institutional_claim_deduped + + - name: int_pharmacy_claim_adr + description: Applying adjustment logic for part D claims. config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: pharmacy_claim_adr - - name: procedure_pivot + - name: int_pharmacy_claim_deduped + description: > + Final de-duplication, transformation logic, and mapping missing fields. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: pharmacy_claim_deduped + + - name: int_physician_claim_adr + description: Applying adjustment logic for part B physician claims. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: physician_claim_adr + + - name: int_physician_claim_deduped + description: > + Final de-duplication, transformation logic, and mapping missing fields. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: physician_claim_deduped + + - name: int_procedure_deduped + description: Dedupe procedures before pivot. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: procedure_deduped + + - name: int_procedure_pivot description: > Pivot of procedures from long (1 row/procedure) to wide (multiple procedure per row). config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: procedure_pivot + + - name: int_revenue_center_deduped + description: Dedupe revenue center detail. + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_int_input_layer{% else %}_int_input_layer{%- endif -%} + alias: revenue_center_deduped -### staging + ### staging - name: stg_beneficiary_demographics - description: Staging model for beneficiary demographics. + description: Staging model for beneficiary demographics (CCLF8). config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: beneficiary_demographics + + - name: stg_beneficiary_xref + description: Staging model for beneficiary cross-reference (CCLF9). + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: beneficiary_xref + + - name: stg_enrollment + description: > + Staging model for custom enrollment data (e.g. attribution, beneficiary + alignment, etc.). + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: enrollment - name: stg_parta_claims_header - description: Staging model for parta claims header. + description: Staging model for parta claims header (CCLF1). config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: parta_claims_header - name: stg_parta_claims_revenue_center_detail - description: Staging model for parta claims revenue center detail. + description: Staging model for parta claims revenue center detail (CCLF2). config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: parta_claims_revenue_center_detail - name: stg_parta_diagnosis_code - description: Staging model for parta diagnosis codes used as ref for pivot transformation. + description: > + Staging model for parta diagnosis codes used as ref for pivot + transformation (CCLF4). config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: parta_diagnosis_code - name: stg_parta_procedure_code - description: Staging model for parta procedure codes used as ref for pivot transformation. + description: > + Staging model for parta procedure codes used as ref for pivot + transformation (CCLF3). config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: parta_procedure_code - name: stg_partb_dme - description: Staging model for partb dme. + description: Staging model for partb dme (CCLF6). config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: partb_dme - name: stg_partb_physicians - description: Staging model for partb physicians. + description: Staging model for partb physicians (CCLF5). + config: + schema: | + {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: partb_physicians + + - name: stg_partd_claims + description: Staging model for partd prescription drug information (CCLF7). config: schema: | {%- if var('tuva_schema_prefix',None) != None -%}_{{var('tuva_schema_prefix')}}_stg_input_layer{% else %}_stg_input_layer{%- endif -%} + alias: partd_claims diff --git a/models/_sources.yml b/models/_sources.yml index 826f7b76..ec3f3494 100644 --- a/models/_sources.yml +++ b/models/_sources.yml @@ -5,10 +5,13 @@ sources: database: "{{ var('input_database') }}" schema: "{{ var('input_schema') }}" tables: - - name: beneficiary_demographics - - name: parta_claims_header - - name: parta_claims_revenue_center_detail - - name: parta_diagnosis_code - - name: parta_procedure_code - - name: partb_physicians - - name: partb_dme \ No newline at end of file + - name: beneficiary_demographics #CCLF8 + - name: beneficiary_xref #CCLF9 + - name: enrollment #custom input + - name: parta_claims_header #CCLF1 + - name: parta_claims_revenue_center_detail #CCLF2 + - name: parta_diagnosis_code #CCLF4 + - name: parta_procedure_code #CCLF3 + - name: partb_dme #CCLF6 + - name: partb_physicians #CCLF5 + - name: partd_claims #CCLF7 diff --git a/models/final/eligibility.sql b/models/final/eligibility.sql index 1011700e..874a5ac3 100644 --- a/models/final/eligibility.sql +++ b/models/final/eligibility.sql @@ -1,128 +1,76 @@ -/* - This model takes in eligibility data on the member month grain and converts - it to enrollment date spans using row number and lag to account for - continuous enrollment and gaps in coverage. -*/ - +/* prep address details for concat */ with demographics as ( select - bene_mbi_id - , cast(bene_sex_cd as {{ dbt.type_string() }} ) as bene_sex_cd - , cast(bene_race_cd as {{ dbt.type_string() }} ) as bene_race_cd + current_bene_mbi_id + , bene_hic_num + , bene_fips_state_cd + , bene_fips_cnty_cd + , bene_zip_cd , bene_dob - , bene_death_dt - , {{ try_to_cast_date('bene_member_month', 'YYYY-MM-DD') }} as bene_member_month - , bene_dual_stus_cd + , bene_sex_cd + , bene_race_cd , bene_mdcr_stus_cd - , bene_orgnl_entlmt_rsn_cd + , bene_dual_stus_cd + , bene_death_dt + , bene_rng_bgn_dt + , bene_rng_end_dt , bene_1st_name + , bene_midl_name , bene_last_name + , bene_orgnl_entlmt_rsn_cd + , bene_entlmt_buyin_ind + , bene_part_a_enrlmt_bgn_dt + , bene_part_b_enrlmt_bgn_dt , bene_line_1_adr + , case + when bene_line_2_adr is null then '' + else cast({{ dbt.concat(["', '","bene_line_2_adr"]) }} as {{ dbt.type_string() }} ) + end as bene_line_2_adr + , case + when bene_line_3_adr is null then '' + else cast({{ dbt.concat(["', '","bene_line_3_adr"]) }} as {{ dbt.type_string() }} ) + end as bene_line_3_adr + , case + when bene_line_4_adr is null then '' + else cast({{ dbt.concat(["', '","bene_line_4_adr"]) }} as {{ dbt.type_string() }} ) + end as bene_line_4_adr + , case + when bene_line_5_adr is null then '' + else cast({{ dbt.concat(["', '","bene_line_5_adr"]) }} as {{ dbt.type_string() }} ) + end as bene_line_5_adr + , case + when bene_line_6_adr is null then '' + else cast({{ dbt.concat(["', '","bene_line_6_adr"]) }} as {{ dbt.type_string() }} ) + end as bene_line_6_adr , geo_zip_plc_name - , cast(bene_fips_state_cd as {{ dbt.type_string() }} ) as bene_fips_state_cd - , bene_zip_cd - , file_name - , ingest_datetime - from {{ ref('stg_beneficiary_demographics') }} - -) - -, fips_state as ( - - select * from {{ ref('reference_data__ansi_fips_state') }} - -) - -, add_row_num as ( - - select * - , row_number() over ( - partition by bene_mbi_id - order by bene_member_month - ) as row_num - from demographics - -) - -, add_lag_enrollment as ( - - select - bene_mbi_id - , bene_member_month - , row_num - , lag(bene_member_month) over ( - partition by bene_mbi_id - order by row_num - ) as lag_enrollment - from add_row_num - -) - -, calculate_lag_diff as ( - - select - bene_mbi_id - , bene_member_month - , row_num - , lag_enrollment - , {{ datediff('lag_enrollment', 'bene_member_month', 'month') }} as lag_diff - from add_lag_enrollment - -) - -, calculate_gaps as ( - - select - bene_mbi_id - , bene_member_month - , row_num - , lag_enrollment - , lag_diff + , geo_usps_state_cd + , geo_zip5_cd , case - when lag_diff > 1 then 1 - else 0 - end as gap_flag - from calculate_lag_diff - -) - -, calculate_groups as ( - - select - bene_mbi_id - , bene_member_month - , row_num - , lag_enrollment - , lag_diff - , gap_flag - , sum(gap_flag) over ( - partition by bene_mbi_id - order by row_num - rows between unbounded preceding and current row - ) as row_group - from calculate_gaps + when geo_zip4_cd is null then '' + else cast({{ dbt.concat(["'-'","geo_zip4_cd"]) }} as {{ dbt.type_string() }} ) + end as geo_zip4_cd + , file_name + , file_date + from {{ ref('int_beneficiary_demographics_deduped') }} ) -, enrollment_span as ( +, enrollment as ( select - bene_mbi_id - , row_group - , min(bene_member_month) as enrollment_start_date - , max(bene_member_month) as enrollment_end_date_max - , {{ last_day('max(bene_member_month)', 'month') }} as enrollment_end_date_last - from calculate_groups - group by bene_mbi_id, row_group + current_bene_mbi_id + , cast(enrollment_start_date as date) as enrollment_start_date + , cast(enrollment_end_date as date) as enrollment_end_date + from {{ ref('int_enrollment') }} ) , joined as ( select - cast(enrollment_span.bene_mbi_id as {{ dbt.type_string() }} ) as patient_id - , cast(enrollment_span.bene_mbi_id as {{ dbt.type_string() }} ) as member_id + cast(demographics.current_bene_mbi_id as {{ dbt.type_string() }} ) as patient_id + , cast(demographics.current_bene_mbi_id as {{ dbt.type_string() }} ) as member_id , cast(null as {{ dbt.type_string() }} ) as subscriber_id , case demographics.bene_sex_cd when '0' then 'unknown' @@ -144,32 +92,48 @@ with demographics as ( when demographics.bene_death_dt is null then 0 else 1 end as integer) as death_flag - , enrollment_span.enrollment_start_date - , enrollment_span.enrollment_end_date_last as enrollment_end_date + , cast(enrollment.enrollment_start_date as date) as enrollment_start_date + , case + when enrollment.enrollment_end_date >= cast({{ dbt.current_timestamp() }} as date) + then {{ last_day(dbt.current_timestamp(), 'month') }} + when enrollment.enrollment_end_date is null then {{ last_day(dbt.current_timestamp(), 'month') }} + else cast(enrollment.enrollment_end_date as date) + end as enrollment_end_date , 'medicare' as payer , 'medicare' as payer_type - , 'medicare' as plan - , cast(bene_orgnl_entlmt_rsn_cd as {{ dbt.type_string() }} ) as original_reason_entitlement_code + , 'medicare' as {{ the_tuva_project.quote_column('plan') }} + , cast(demographics.bene_orgnl_entlmt_rsn_cd as {{ dbt.type_string() }} ) as original_reason_entitlement_code , cast(demographics.bene_dual_stus_cd as {{ dbt.type_string() }} ) as dual_status_code , cast(demographics.bene_mdcr_stus_cd as {{ dbt.type_string() }} ) as medicare_status_code , cast(demographics.bene_1st_name as {{ dbt.type_string() }} ) as first_name , cast(demographics.bene_last_name as {{ dbt.type_string() }} ) as last_name , cast(null as {{ dbt.type_string() }} ) as social_security_number - , cast(null as {{ dbt.type_string() }} ) as subscriber_relation - , cast(demographics.bene_line_1_adr as {{ dbt.type_string() }} ) as address + , cast('self' as {{ dbt.type_string() }} ) as subscriber_relation + , {{ dbt.concat( + [ + "demographics.bene_line_1_adr", + "demographics.bene_line_2_adr", + "demographics.bene_line_3_adr", + "demographics.bene_line_4_adr", + "demographics.bene_line_5_adr", + "demographics.bene_line_6_adr" + ] + ) }} as address , cast(demographics.geo_zip_plc_name as {{ dbt.type_string() }} ) as city - , cast(fips_state.ansi_fips_state_name as {{ dbt.type_string() }} ) as state - , cast(demographics.bene_zip_cd as {{ dbt.type_string() }} ) as zip_code + , cast(demographics.geo_usps_state_cd as {{ dbt.type_string() }} ) as state + , {{ dbt.concat( + [ + "demographics.geo_zip5_cd", + "demographics.geo_zip4_cd" + ] + ) }} as zip_code , cast(NULL as {{ dbt.type_string() }} ) as phone , 'medicare cclf' as data_source , cast(demographics.file_name as {{ dbt.type_string() }} ) as file_name - , cast(demographics.ingest_datetime as {{ dbt.type_timestamp() }} ) as ingest_datetime - from enrollment_span - left join demographics - on enrollment_span.bene_mbi_id = demographics.bene_mbi_id - and enrollment_span.enrollment_end_date_max = demographics.bene_member_month - left join fips_state - on demographics.bene_fips_state_cd = fips_state.ansi_fips_state_code + , cast(demographics.file_date as {{ dbt.type_timestamp() }} ) as ingest_datetime + from demographics + left join enrollment + on demographics.current_bene_mbi_id = enrollment.current_bene_mbi_id ) @@ -186,7 +150,7 @@ select , enrollment_end_date , payer , payer_type - , plan + , {{ the_tuva_project.quote_column('plan') }} , original_reason_entitlement_code , dual_status_code , medicare_status_code diff --git a/models/final/medical_claim.sql b/models/final/medical_claim.sql index 9fb3143e..a9632e26 100644 --- a/models/final/medical_claim.sql +++ b/models/final/medical_claim.sql @@ -1,10 +1,14 @@ with unioned as ( - select * from {{ ref('institutional_claims')}} - union all - select * from {{ ref('physician_claims')}} - union all - select * from {{ ref('dme_claims')}} + {{ dbt_utils.union_relations( + + relations=[ + ref('int_dme_claim_deduped') + , ref('int_institutional_claim_deduped') + , ref('int_physician_claim_deduped') + ] + + ) }} ) @@ -15,7 +19,7 @@ select , patient_id , member_id , payer - , plan + , {{ the_tuva_project.quote_column('plan') }} , claim_start_date , claim_end_date , claim_line_start_date diff --git a/models/final/pharmacy_claim.sql b/models/final/pharmacy_claim.sql index 64ee6828..7380cd9a 100644 --- a/models/final/pharmacy_claim.sql +++ b/models/final/pharmacy_claim.sql @@ -1,26 +1,63 @@ +with deduped_claims as ( + + select * from {{ ref('int_pharmacy_claim_deduped') }} + +) + +, data_types as ( + + select + cast(claim_id as {{ dbt.type_string() }}) as claim_id + , cast(claim_line_number as integer) as claim_line_number + , cast(patient_id as {{ dbt.type_string() }}) as patient_id + , cast(member_id as {{ dbt.type_string() }}) as member_id + , cast(payer as {{ dbt.type_string() }}) as payer + , cast({{ the_tuva_project.quote_column('plan') }} as {{ dbt.type_string() }}) as {{ the_tuva_project.quote_column('plan') }} + , cast(prescribing_provider_npi as {{ dbt.type_string() }}) as prescribing_provider_npi + , cast(dispensing_provider_npi as {{ dbt.type_string() }}) as dispensing_provider_npi + , cast(dispensing_date as date) as dispensing_date + , cast(ndc_code as {{ dbt.type_string() }}) as ndc_code + , cast(quantity as integer) as quantity + , cast(days_supply as integer) as days_supply + , cast(refills as integer) as refills + , cast(paid_date as date) as paid_date + , {{ cast_numeric('paid_amount') }} as paid_amount + , {{ cast_numeric('allowed_amount') }} as allowed_amount + , {{ cast_numeric('charge_amount') }} as charge_amount + , {{ cast_numeric('coinsurance_amount') }} as coinsurance_amount + , {{ cast_numeric('copayment_amount') }} as copayment_amount + , {{ cast_numeric('deductible_amount') }} as deductible_amount + , cast(in_network_flag as integer) as in_network_flag + , cast(data_source as {{ dbt.type_string() }}) as data_source + , cast(file_name as {{ dbt.type_string() }}) as file_name + , cast(ingest_datetime as {{ dbt.type_timestamp() }}) as ingest_datetime + from deduped_claims + +) + select - cast(null as {{ dbt.type_string() }}) as claim_id - , cast(null as {{ dbt.type_string() }}) as claim_line_number - , cast(null as {{ dbt.type_string() }}) as patient_id - , cast(null as {{ dbt.type_string() }}) as member_id - , cast(null as {{ dbt.type_string() }}) as payer - , cast(null as {{ dbt.type_string() }}) as plan - , cast(null as {{ dbt.type_string() }}) as prescribing_provider_npi - , cast(null as {{ dbt.type_string() }}) as dispensing_provider_npi - , cast(null as date ) as dispensing_date - , cast(null as {{ dbt.type_string() }}) as ndc_code - , cast(null as int) as quantity - , cast(null as int) as days_supply - , cast(null as int) as refills - , cast(null as date) as paid_date - , cast(null as numeric) as paid_amount - , cast(null as numeric) as allowed_amount - , cast(null as numeric) as charge_amount - , cast(null as numeric) as coinsurance_amount - , cast(null as numeric) as copayment_amount - , cast(null as numeric) as deductible_amount - , cast(null as int) as in_network_flag - , cast(null as {{ dbt.type_string() }}) as data_source - , cast(null as {{ dbt.type_string() }}) as file_name - , cast(null as {{ dbt.type_timestamp() }}) as ingest_datetime -limit 0 \ No newline at end of file + claim_id + , claim_line_number + , patient_id + , member_id + , payer + , {{ the_tuva_project.quote_column('plan') }} + , prescribing_provider_npi + , dispensing_provider_npi + , dispensing_date + , ndc_code + , quantity + , days_supply + , refills + , paid_date + , paid_amount + , allowed_amount + , charge_amount + , coinsurance_amount + , copayment_amount + , deductible_amount + , in_network_flag + , data_source + , file_name + , ingest_datetime +from data_types \ No newline at end of file diff --git a/models/intermediate/dme_claims.sql b/models/intermediate/dme_claims.sql deleted file mode 100644 index bfec3265..00000000 --- a/models/intermediate/dme_claims.sql +++ /dev/null @@ -1,152 +0,0 @@ -select - cast(cur_clm_uniq_id as {{ dbt.type_string() }} ) as claim_id - , cast(clm_line_num as integer) as claim_line_number - , 'professional' as claim_type - , cast(bene_mbi_id as {{ dbt.type_string() }} ) as patient_id - , cast(bene_mbi_id as {{ dbt.type_string() }} ) as member_id - , 'medicare' as payer - , 'medicare' as plan - , {{ try_to_cast_date('clm_from_dt', 'YYYY-MM-DD') }} as claim_start_date - , {{ try_to_cast_date('clm_thru_dt', 'YYYY-MM-DD') }} as claim_end_date - , {{ try_to_cast_date('clm_line_from_dt', 'YYYY-MM-DD') }} as claim_line_start_date - , {{ try_to_cast_date('clm_line_thru_dt', 'YYYY-MM-DD') }} as claim_line_end_date - , cast(NULL as date) as admission_date - , cast(NULL as date) as discharge_date - , cast(NULL as {{ dbt.type_string() }} ) as admit_source_code - , cast(NULL as {{ dbt.type_string() }} ) as admit_type_code - , cast(NULL as {{ dbt.type_string() }} ) as discharge_disposition_code - , cast(clm_pos_cd as {{ dbt.type_string() }} ) as place_of_service_code - , cast(NULL as {{ dbt.type_string() }} ) as bill_type_code - , cast(NULL as {{ dbt.type_string() }} ) as ms_drg_code - , cast(NULL as {{ dbt.type_string() }} ) as apr_drg_code - , cast(NULL as {{ dbt.type_string() }} ) as revenue_center_code - , cast(NULL as integer) as service_unit_quantity - , cast(clm_line_hcpcs_cd as {{ dbt.type_string() }} ) as hcpcs_code - , cast(NULL as {{ dbt.type_string() }} ) as hcpcs_modifier_1 - , cast(NULL as {{ dbt.type_string() }} ) as hcpcs_modifier_2 - , cast(NULL as {{ dbt.type_string() }} ) as hcpcs_modifier_3 - , cast(NULL as {{ dbt.type_string() }} ) as hcpcs_modifier_4 - , cast(NULL as {{ dbt.type_string() }} ) as hcpcs_modifier_5 - , cast(ordrg_prvdr_npi_num as {{ dbt.type_string() }} ) as rendering_npi - , cast(NULL as {{ dbt.type_string() }} ) as rendering_tin - , cast(payto_prvdr_npi_num as {{ dbt.type_string() }} ) as billing_npi - , cast(NULL as {{ dbt.type_string() }} ) as billing_tin - , cast(NULL as {{ dbt.type_string() }} ) as facility_npi - , cast(NULL as date) as paid_date - , case - when cast(clm_adjsmt_type_cd as {{ dbt.type_string() }} ) = '1' then {{ cast_numeric('clm_line_cvrd_pd_amt') }} * -1 - else {{ cast_numeric('clm_line_cvrd_pd_amt') }} - end as paid_amount - , {{ cast_numeric('clm_line_alowd_chrg_amt') }} as allowed_amount - , {{ cast_numeric('clm_line_alowd_chrg_amt') }} as charge_amount - , {{ cast_numeric('NULL') }} as coinsurance_amount - , {{ cast_numeric('NULL') }} as copayment_amount - , {{ cast_numeric('NULL') }} as deductible_amount - , {{ cast_numeric('NULL') }} as total_cost_amount - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_type - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_1 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_2 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_3 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_4 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_5 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_6 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_7 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_8 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_9 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_10 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_11 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_12 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_13 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_14 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_15 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_16 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_17 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_18 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_19 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_20 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_21 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_22 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_23 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_24 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_25 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_1 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_2 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_3 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_4 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_5 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_6 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_7 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_8 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_9 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_10 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_11 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_12 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_13 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_14 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_15 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_16 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_17 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_18 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_19 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_20 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_21 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_22 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_23 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_24 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_25 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_type - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_1 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_2 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_3 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_4 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_5 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_6 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_7 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_8 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_9 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_10 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_11 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_12 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_13 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_14 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_15 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_16 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_17 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_18 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_19 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_20 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_21 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_22 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_23 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_24 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_25 - , cast(NULL as date) as procedure_date_1 - , cast(NULL as date) as procedure_date_2 - , cast(NULL as date) as procedure_date_3 - , cast(NULL as date) as procedure_date_4 - , cast(NULL as date) as procedure_date_5 - , cast(NULL as date) as procedure_date_6 - , cast(NULL as date) as procedure_date_7 - , cast(NULL as date) as procedure_date_8 - , cast(NULL as date) as procedure_date_9 - , cast(NULL as date) as procedure_date_10 - , cast(NULL as date) as procedure_date_11 - , cast(NULL as date) as procedure_date_12 - , cast(NULL as date) as procedure_date_13 - , cast(NULL as date) as procedure_date_14 - , cast(NULL as date) as procedure_date_15 - , cast(NULL as date) as procedure_date_16 - , cast(NULL as date) as procedure_date_17 - , cast(NULL as date) as procedure_date_18 - , cast(NULL as date) as procedure_date_19 - , cast(NULL as date) as procedure_date_20 - , cast(NULL as date) as procedure_date_21 - , cast(NULL as date) as procedure_date_22 - , cast(NULL as date) as procedure_date_23 - , cast(NULL as date) as procedure_date_24 - , cast(NULL as date) as procedure_date_25 - , cast(1 as int) as in_network_flag - , 'medicare cclf' as data_source - , cast(file_name as {{ dbt.type_string() }} ) as file_name - , cast(ingest_datetime as {{ dbt.type_timestamp() }} ) as ingest_datetime -from {{ ref('stg_partb_dme') }} \ No newline at end of file diff --git a/models/intermediate/institutional_claims.sql b/models/intermediate/institutional_claims.sql deleted file mode 100644 index f2bd19e2..00000000 --- a/models/intermediate/institutional_claims.sql +++ /dev/null @@ -1,211 +0,0 @@ -with claim_line as ( - - select distinct - a.cur_clm_uniq_id as claim_id - , b.clm_line_num as claim_line_number - , row_number() over(partition by a.cur_clm_uniq_id order by b.clm_line_num) as claim_row_number - from {{ ref('stg_parta_claims_header') }} as a - left join {{ ref('stg_parta_claims_revenue_center_detail') }} as b - on a.cur_clm_uniq_id = b.cur_clm_uniq_id - -) - -, add_header_paid_amount as ( - - select - a.claim_id - , a.claim_line_number - , a.claim_row_number - , b.clm_pmt_amt as paid_amount - from claim_line as a - inner join {{ ref('stg_parta_claims_header') }} as b - on a.claim_id = b.cur_clm_uniq_id - where a.claim_row_number = 1 - -) - -, claim_line_a as ( - - select - a.claim_id - , a.claim_line_number - , a.claim_row_number - , b.paid_amount - from claim_line as a - left join add_header_paid_amount as b - on a.claim_id = b.claim_id - and a.claim_row_number = b.claim_row_number - -) - -select - cast(a.claim_id as {{ dbt.type_string() }} ) as claim_id - , cast(a.claim_line_number as integer) as claim_line_number - , 'institutional' as claim_type - , cast(h.bene_mbi_id as {{ dbt.type_string() }} ) as patient_id - , cast(h.bene_mbi_id as {{ dbt.type_string() }} ) as member_id - , 'medicare' as payer - , 'medicare' as plan - , {{ try_to_cast_date('h.clm_from_dt', 'YYYY-MM-DD') }} as claim_start_date - , {{ try_to_cast_date('h.clm_thru_dt', 'YYYY-MM-DD') }} as claim_end_date - , cast(NULL as date) as claim_line_start_date - , cast(NULL as date) as claim_line_end_date - , cast(NULL as date) as admission_date - , cast(NULL as date) as discharge_date - , cast(h.clm_admsn_src_cd as {{ dbt.type_string() }} ) as admit_source_code - , cast(h.clm_admsn_type_cd as {{ dbt.type_string() }} ) as admit_type_code - , lpad(cast(h.bene_ptnt_stus_cd as {{ dbt.type_string() }} ),2, '0') as discharge_disposition_code - , cast(NULL as {{ dbt.type_string() }} ) as place_of_service_code - , cast(h.clm_bill_fac_type_cd as {{ dbt.type_string() }} ) - || cast(h.clm_bill_clsfctn_cd as {{ dbt.type_string() }} ) - || cast(h.clm_bill_freq_cd as {{ dbt.type_string() }} ) - as bill_type_code - , cast(h.dgns_drg_cd as {{ dbt.type_string() }} ) as ms_drg_code - , cast(null as {{ dbt.type_string() }} ) as apr_drg_code - , lpad(cast(d.clm_line_prod_rev_ctr_cd as {{ dbt.type_string() }} ),4,'0') as revenue_center_code - , cast(d.clm_line_srvc_unit_qty as integer) as service_unit_quantity - , cast(d.clm_line_hcpcs_cd as {{ dbt.type_string() }} ) as hcpcs_code - , cast(d.hcpcs_1_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_1 - , cast(d.hcpcs_2_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_2 - , cast(d.hcpcs_3_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_3 - , cast(d.hcpcs_4_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_4 - , cast(d.hcpcs_5_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_5 - , cast(h.atndg_prvdr_npi_num as {{ dbt.type_string() }} ) as rendering_npi - , cast(NULL as {{ dbt.type_string() }} ) as rendering_tin - , cast(NULL as {{ dbt.type_string() }} ) as billing_npi - , cast(NULL as {{ dbt.type_string() }} ) as billing_tin - , cast(h.fac_prvdr_npi_num as {{ dbt.type_string() }} ) as facility_npi - , cast(NULL as date) as paid_date - , case - when cast(h.clm_adjsmt_type_cd as {{ dbt.type_string() }} ) = '1' then {{ cast_numeric('a.paid_amount') }} * -1 - else {{ cast_numeric('a.paid_amount') }} - end as paid_amount - , {{ cast_numeric('NULL') }} as allowed_amount - , {{ cast_numeric('h.clm_mdcr_instnl_tot_chrg_amt') }} as charge_amount - , {{ cast_numeric('NULL') }} as coinsurance_amount - , {{ cast_numeric('NULL') }} as copayment_amount - , {{ cast_numeric('NULL') }} as deductible_amount - , {{ cast_numeric('NULL') }} as total_cost_amount - , case - when cast(dx.dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) = '0' then 'icd-10-cm' - when cast(dx.dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) = '9' then 'icd-9-cm' - else cast(dx.dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) - end as diagnosis_code_type - , cast(dx.diagnosis_code_1 as {{ dbt.type_string() }} ) as diagnosis_code_1 - , cast(dx.diagnosis_code_2 as {{ dbt.type_string() }} ) as diagnosis_code_2 - , cast(dx.diagnosis_code_3 as {{ dbt.type_string() }} ) as diagnosis_code_3 - , cast(dx.diagnosis_code_4 as {{ dbt.type_string() }} ) as diagnosis_code_4 - , cast(dx.diagnosis_code_5 as {{ dbt.type_string() }} ) as diagnosis_code_5 - , cast(dx.diagnosis_code_6 as {{ dbt.type_string() }} ) as diagnosis_code_6 - , cast(dx.diagnosis_code_7 as {{ dbt.type_string() }} ) as diagnosis_code_7 - , cast(dx.diagnosis_code_8 as {{ dbt.type_string() }} ) as diagnosis_code_8 - , cast(dx.diagnosis_code_9 as {{ dbt.type_string() }} ) as diagnosis_code_9 - , cast(dx.diagnosis_code_10 as {{ dbt.type_string() }} ) as diagnosis_code_10 - , cast(dx.diagnosis_code_11 as {{ dbt.type_string() }} ) as diagnosis_code_11 - , cast(dx.diagnosis_code_12 as {{ dbt.type_string() }} ) as diagnosis_code_12 - , cast(dx.diagnosis_code_13 as {{ dbt.type_string() }} ) as diagnosis_code_13 - , cast(dx.diagnosis_code_14 as {{ dbt.type_string() }} ) as diagnosis_code_14 - , cast(dx.diagnosis_code_15 as {{ dbt.type_string() }} ) as diagnosis_code_15 - , cast(dx.diagnosis_code_16 as {{ dbt.type_string() }} ) as diagnosis_code_16 - , cast(dx.diagnosis_code_17 as {{ dbt.type_string() }} ) as diagnosis_code_17 - , cast(dx.diagnosis_code_18 as {{ dbt.type_string() }} ) as diagnosis_code_18 - , cast(dx.diagnosis_code_19 as {{ dbt.type_string() }} ) as diagnosis_code_19 - , cast(dx.diagnosis_code_20 as {{ dbt.type_string() }} ) as diagnosis_code_20 - , cast(dx.diagnosis_code_21 as {{ dbt.type_string() }} ) as diagnosis_code_21 - , cast(dx.diagnosis_code_22 as {{ dbt.type_string() }} ) as diagnosis_code_22 - , cast(dx.diagnosis_code_23 as {{ dbt.type_string() }} ) as diagnosis_code_23 - , cast(dx.diagnosis_code_24 as {{ dbt.type_string() }} ) as diagnosis_code_24 - , cast(dx.diagnosis_code_25 as {{ dbt.type_string() }} ) as diagnosis_code_25 - , cast(dx.diagnosis_poa_1 as {{ dbt.type_string() }} ) as diagnosis_poa_1 - , cast(dx.diagnosis_poa_2 as {{ dbt.type_string() }} ) as diagnosis_poa_2 - , cast(dx.diagnosis_poa_3 as {{ dbt.type_string() }} ) as diagnosis_poa_3 - , cast(dx.diagnosis_poa_4 as {{ dbt.type_string() }} ) as diagnosis_poa_4 - , cast(dx.diagnosis_poa_5 as {{ dbt.type_string() }} ) as diagnosis_poa_5 - , cast(dx.diagnosis_poa_6 as {{ dbt.type_string() }} ) as diagnosis_poa_6 - , cast(dx.diagnosis_poa_7 as {{ dbt.type_string() }} ) as diagnosis_poa_7 - , cast(dx.diagnosis_poa_8 as {{ dbt.type_string() }} ) as diagnosis_poa_8 - , cast(dx.diagnosis_poa_9 as {{ dbt.type_string() }} ) as diagnosis_poa_9 - , cast(dx.diagnosis_poa_10 as {{ dbt.type_string() }} ) as diagnosis_poa_10 - , cast(dx.diagnosis_poa_11 as {{ dbt.type_string() }} ) as diagnosis_poa_11 - , cast(dx.diagnosis_poa_12 as {{ dbt.type_string() }} ) as diagnosis_poa_12 - , cast(dx.diagnosis_poa_13 as {{ dbt.type_string() }} ) as diagnosis_poa_13 - , cast(dx.diagnosis_poa_14 as {{ dbt.type_string() }} ) as diagnosis_poa_14 - , cast(dx.diagnosis_poa_15 as {{ dbt.type_string() }} ) as diagnosis_poa_15 - , cast(dx.diagnosis_poa_16 as {{ dbt.type_string() }} ) as diagnosis_poa_16 - , cast(dx.diagnosis_poa_17 as {{ dbt.type_string() }} ) as diagnosis_poa_17 - , cast(dx.diagnosis_poa_18 as {{ dbt.type_string() }} ) as diagnosis_poa_18 - , cast(dx.diagnosis_poa_19 as {{ dbt.type_string() }} ) as diagnosis_poa_19 - , cast(dx.diagnosis_poa_20 as {{ dbt.type_string() }} ) as diagnosis_poa_20 - , cast(dx.diagnosis_poa_21 as {{ dbt.type_string() }} ) as diagnosis_poa_21 - , cast(dx.diagnosis_poa_22 as {{ dbt.type_string() }} ) as diagnosis_poa_22 - , cast(dx.diagnosis_poa_23 as {{ dbt.type_string() }} ) as diagnosis_poa_23 - , cast(dx.diagnosis_poa_24 as {{ dbt.type_string() }} ) as diagnosis_poa_24 - , cast(dx.diagnosis_poa_25 as {{ dbt.type_string() }} ) as diagnosis_poa_25 - , case - when cast(px.dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) = '0' then 'icd-10-pcs' - when cast(px.dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) = '9' then 'icd-9-pcs' - else cast(px.dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) end procedure_code_type - , cast(px.procedure_code_1 as {{ dbt.type_string() }} ) as procedure_code_1 - , cast(px.procedure_code_2 as {{ dbt.type_string() }} ) as procedure_code_2 - , cast(px.procedure_code_3 as {{ dbt.type_string() }} ) as procedure_code_3 - , cast(px.procedure_code_4 as {{ dbt.type_string() }} ) as procedure_code_4 - , cast(px.procedure_code_5 as {{ dbt.type_string() }} ) as procedure_code_5 - , cast(px.procedure_code_6 as {{ dbt.type_string() }} ) as procedure_code_6 - , cast(px.procedure_code_7 as {{ dbt.type_string() }} ) as procedure_code_7 - , cast(px.procedure_code_8 as {{ dbt.type_string() }} ) as procedure_code_8 - , cast(px.procedure_code_9 as {{ dbt.type_string() }} ) as procedure_code_9 - , cast(px.procedure_code_10 as {{ dbt.type_string() }} ) as procedure_code_10 - , cast(px.procedure_code_11 as {{ dbt.type_string() }} ) as procedure_code_11 - , cast(px.procedure_code_12 as {{ dbt.type_string() }} ) as procedure_code_12 - , cast(px.procedure_code_13 as {{ dbt.type_string() }} ) as procedure_code_13 - , cast(px.procedure_code_14 as {{ dbt.type_string() }} ) as procedure_code_14 - , cast(px.procedure_code_15 as {{ dbt.type_string() }} ) as procedure_code_15 - , cast(px.procedure_code_16 as {{ dbt.type_string() }} ) as procedure_code_16 - , cast(px.procedure_code_17 as {{ dbt.type_string() }} ) as procedure_code_17 - , cast(px.procedure_code_18 as {{ dbt.type_string() }} ) as procedure_code_18 - , cast(px.procedure_code_19 as {{ dbt.type_string() }} ) as procedure_code_19 - , cast(px.procedure_code_20 as {{ dbt.type_string() }} ) as procedure_code_20 - , cast(px.procedure_code_21 as {{ dbt.type_string() }} ) as procedure_code_21 - , cast(px.procedure_code_22 as {{ dbt.type_string() }} ) as procedure_code_22 - , cast(px.procedure_code_23 as {{ dbt.type_string() }} ) as procedure_code_23 - , cast(px.procedure_code_24 as {{ dbt.type_string() }} ) as procedure_code_24 - , cast(px.procedure_code_25 as {{ dbt.type_string() }} ) as procedure_code_25 - , {{ try_to_cast_date('px.procedure_date_1', 'YYYY-MM-DD') }} as procedure_date_1 - , {{ try_to_cast_date('px.procedure_date_2', 'YYYY-MM-DD') }} as procedure_date_2 - , {{ try_to_cast_date('px.procedure_date_3', 'YYYY-MM-DD') }} as procedure_date_3 - , {{ try_to_cast_date('px.procedure_date_4', 'YYYY-MM-DD') }} as procedure_date_4 - , {{ try_to_cast_date('px.procedure_date_5', 'YYYY-MM-DD') }} as procedure_date_5 - , {{ try_to_cast_date('px.procedure_date_6', 'YYYY-MM-DD') }} as procedure_date_6 - , {{ try_to_cast_date('px.procedure_date_7', 'YYYY-MM-DD') }} as procedure_date_7 - , {{ try_to_cast_date('px.procedure_date_8', 'YYYY-MM-DD') }} as procedure_date_8 - , {{ try_to_cast_date('px.procedure_date_9', 'YYYY-MM-DD') }} as procedure_date_9 - , {{ try_to_cast_date('px.procedure_date_10', 'YYYY-MM-DD') }} as procedure_date_10 - , {{ try_to_cast_date('px.procedure_date_11', 'YYYY-MM-DD') }} as procedure_date_11 - , {{ try_to_cast_date('px.procedure_date_12', 'YYYY-MM-DD') }} as procedure_date_12 - , {{ try_to_cast_date('px.procedure_date_13', 'YYYY-MM-DD') }} as procedure_date_13 - , {{ try_to_cast_date('px.procedure_date_14', 'YYYY-MM-DD') }} as procedure_date_14 - , {{ try_to_cast_date('px.procedure_date_15', 'YYYY-MM-DD') }} as procedure_date_15 - , {{ try_to_cast_date('px.procedure_date_16', 'YYYY-MM-DD') }} as procedure_date_16 - , {{ try_to_cast_date('px.procedure_date_17', 'YYYY-MM-DD') }} as procedure_date_17 - , {{ try_to_cast_date('px.procedure_date_18', 'YYYY-MM-DD') }} as procedure_date_18 - , {{ try_to_cast_date('px.procedure_date_19', 'YYYY-MM-DD') }} as procedure_date_19 - , {{ try_to_cast_date('px.procedure_date_20', 'YYYY-MM-DD') }} as procedure_date_20 - , {{ try_to_cast_date('px.procedure_date_21', 'YYYY-MM-DD') }} as procedure_date_21 - , {{ try_to_cast_date('px.procedure_date_22', 'YYYY-MM-DD') }} as procedure_date_22 - , {{ try_to_cast_date('px.procedure_date_23', 'YYYY-MM-DD') }} as procedure_date_23 - , {{ try_to_cast_date('px.procedure_date_24', 'YYYY-MM-DD') }} as procedure_date_24 - , {{ try_to_cast_date('px.procedure_date_25', 'YYYY-MM-DD') }} as procedure_date_25 - , cast(1 as int) as in_network_flag - , 'medicare cclf' as data_source - , cast(h.file_name as {{ dbt.type_string() }} ) as file_name - , cast(h.ingest_datetime as {{ dbt.type_timestamp() }} ) as ingest_datetime -from claim_line_a as a - left join {{ ref('stg_parta_claims_header') }} as h - on a.claim_id = h.cur_clm_uniq_id - left join {{ ref('stg_parta_claims_revenue_center_detail') }} as d - on a.claim_id = d.cur_clm_uniq_id - and a.claim_line_number = d.clm_line_num - left join {{ ref('procedure_pivot') }} as px - on cast(a.claim_id as {{ dbt.type_string() }} ) = cast(px.cur_clm_uniq_id as {{ dbt.type_string() }} ) - left join {{ ref('diagnosis_pivot') }} as dx - on cast(a.claim_id as {{ dbt.type_string() }} ) = cast(dx.cur_clm_uniq_id as {{ dbt.type_string() }} ) \ No newline at end of file diff --git a/models/intermediate/int_beneficiary_demographics_deduped.sql b/models/intermediate/int_beneficiary_demographics_deduped.sql new file mode 100644 index 00000000..2da5a122 --- /dev/null +++ b/models/intermediate/int_beneficiary_demographics_deduped.sql @@ -0,0 +1,213 @@ +with staged_data as ( + + select + bene_mbi_id + , bene_hic_num + , bene_fips_state_cd + , bene_fips_cnty_cd + , bene_zip_cd + , bene_dob + , bene_sex_cd + , bene_race_cd + , bene_age + , bene_mdcr_stus_cd + , bene_dual_stus_cd + , bene_death_dt + , bene_rng_bgn_dt + , bene_rng_end_dt + , bene_1st_name + , bene_midl_name + , bene_last_name + , bene_orgnl_entlmt_rsn_cd + , bene_entlmt_buyin_ind + , bene_part_a_enrlmt_bgn_dt + , bene_part_b_enrlmt_bgn_dt + , bene_line_1_adr + , bene_line_2_adr + , bene_line_3_adr + , bene_line_4_adr + , bene_line_5_adr + , bene_line_6_adr + , geo_zip_plc_name + , geo_usps_state_cd + , geo_zip5_cd + , geo_zip4_cd + , file_name + , file_date + from {{ ref('stg_beneficiary_demographics') }} + +) + +/* dedupe full rows that may appear in multiple files */ +, add_row_num as ( + + select *, row_number() over ( + partition by + bene_mbi_id + , bene_hic_num + , bene_fips_state_cd + , bene_fips_cnty_cd + , bene_zip_cd + , bene_dob + , bene_sex_cd + , bene_race_cd + /*, bene_age*/ --commenting out since it causes duplicates and is not used + , bene_mdcr_stus_cd + , bene_dual_stus_cd + , bene_death_dt + , bene_rng_bgn_dt + , bene_rng_end_dt + , bene_1st_name + , bene_midl_name + , bene_last_name + , bene_orgnl_entlmt_rsn_cd + , bene_entlmt_buyin_ind + , bene_part_a_enrlmt_bgn_dt + , bene_part_b_enrlmt_bgn_dt + , bene_line_1_adr + , bene_line_2_adr + , bene_line_3_adr + , bene_line_4_adr + , bene_line_5_adr + , bene_line_6_adr + , geo_zip_plc_name + , geo_usps_state_cd + , geo_zip5_cd + , geo_zip4_cd + order by file_date desc + ) as row_num + from staged_data + +) + +, beneficiary_xref as ( + + select + crnt_num + , prvs_num + from {{ ref('int_beneficiary_xref_deduped') }} + +) + +, add_mbi_xref as ( + + select + add_row_num.bene_mbi_id + , coalesce(beneficiary_xref.crnt_num, add_row_num.bene_mbi_id) as current_bene_mbi_id + , add_row_num.bene_hic_num + , add_row_num.bene_fips_state_cd + , add_row_num.bene_fips_cnty_cd + , add_row_num.bene_zip_cd + , add_row_num.bene_dob + , add_row_num.bene_sex_cd + , add_row_num.bene_race_cd + , add_row_num.bene_mdcr_stus_cd + , add_row_num.bene_dual_stus_cd + , add_row_num.bene_death_dt + , add_row_num.bene_rng_bgn_dt + , add_row_num.bene_rng_end_dt + , add_row_num.bene_1st_name + , add_row_num.bene_midl_name + , add_row_num.bene_last_name + , add_row_num.bene_orgnl_entlmt_rsn_cd + , add_row_num.bene_entlmt_buyin_ind + , add_row_num.bene_part_a_enrlmt_bgn_dt + , add_row_num.bene_part_b_enrlmt_bgn_dt + , add_row_num.bene_line_1_adr + , add_row_num.bene_line_2_adr + , add_row_num.bene_line_3_adr + , add_row_num.bene_line_4_adr + , add_row_num.bene_line_5_adr + , add_row_num.bene_line_6_adr + , add_row_num.geo_zip_plc_name + , add_row_num.geo_usps_state_cd + , add_row_num.geo_zip5_cd + , add_row_num.geo_zip4_cd + , add_row_num.file_name + , add_row_num.file_date + from add_row_num + left join beneficiary_xref + on add_row_num.bene_mbi_id = beneficiary_xref.prvs_num + where row_num = 1 + +) + +, get_latest_mbi as ( + + select + bene_mbi_id + , current_bene_mbi_id + , bene_hic_num + , bene_fips_state_cd + , bene_fips_cnty_cd + , bene_zip_cd + , bene_dob + , bene_sex_cd + , bene_race_cd + , bene_mdcr_stus_cd + , bene_dual_stus_cd + , bene_death_dt + , bene_rng_bgn_dt + , bene_rng_end_dt + , bene_1st_name + , bene_midl_name + , bene_last_name + , bene_orgnl_entlmt_rsn_cd + , bene_entlmt_buyin_ind + , bene_part_a_enrlmt_bgn_dt + , bene_part_b_enrlmt_bgn_dt + , bene_line_1_adr + , bene_line_2_adr + , bene_line_3_adr + , bene_line_4_adr + , bene_line_5_adr + , bene_line_6_adr + , geo_zip_plc_name + , geo_usps_state_cd + , geo_zip5_cd + , geo_zip4_cd + , file_name + , file_date + , row_number() over ( + partition by current_bene_mbi_id + order by file_date desc + ) as row_num + from add_mbi_xref + +) + +select + current_bene_mbi_id + , bene_hic_num + , bene_fips_state_cd + , bene_fips_cnty_cd + , bene_zip_cd + , bene_dob + , bene_sex_cd + , bene_race_cd + , bene_mdcr_stus_cd + , bene_dual_stus_cd + , bene_death_dt + , bene_rng_bgn_dt + , bene_rng_end_dt + , bene_1st_name + , bene_midl_name + , bene_last_name + , bene_orgnl_entlmt_rsn_cd + , bene_entlmt_buyin_ind + , bene_part_a_enrlmt_bgn_dt + , bene_part_b_enrlmt_bgn_dt + , bene_line_1_adr + , bene_line_2_adr + , bene_line_3_adr + , bene_line_4_adr + , bene_line_5_adr + , bene_line_6_adr + , geo_zip_plc_name + , geo_usps_state_cd + , geo_zip5_cd + , geo_zip4_cd + , file_name + , file_date +from get_latest_mbi +where row_num = 1 \ No newline at end of file diff --git a/models/intermediate/int_beneficiary_xref_deduped.sql b/models/intermediate/int_beneficiary_xref_deduped.sql new file mode 100644 index 00000000..184d3f65 --- /dev/null +++ b/models/intermediate/int_beneficiary_xref_deduped.sql @@ -0,0 +1,55 @@ +with staged_data as ( + + select + hicn_mbi_xref_ind + , crnt_num + , prvs_num + , prvs_id_efctv_dt + , prvs_id_obslt_dt + , bene_rrb_num + , file_name + , file_date + from {{ ref('stg_beneficiary_xref') }} + +) + +/* window over previous MBI to get latest current MBI */ +, add_row_num as ( + + select *, row_number() over ( + partition by prvs_num + order by file_date desc, prvs_id_efctv_dt desc + ) as row_num + from staged_data + +) + +/* + check if the current MBI is listed as a previous MBI and + get its latest current MBI +*/ +, check_crnt_num as ( + select + a.file_date + , a.prvs_num + , a.crnt_num + , b.file_date as b_file_date + , b.prvs_num as b_prvs_num + , b.crnt_num as b_crnt_num + , case + when b.crnt_num is not null and b.file_date > a.file_date + then b.crnt_num + else a.crnt_num + end as final_mbi + from add_row_num as a + left join add_row_num as b + on a.crnt_num = b.prvs_num + and b.prvs_num <> b.crnt_num + where a.row_num = 1 + +) + +select + prvs_num + , final_mbi as crnt_num +from check_crnt_num \ No newline at end of file diff --git a/models/intermediate/int_diagnosis_deduped.sql b/models/intermediate/int_diagnosis_deduped.sql new file mode 100644 index 00000000..35a9fa73 --- /dev/null +++ b/models/intermediate/int_diagnosis_deduped.sql @@ -0,0 +1,66 @@ +with staged_data as ( + + select + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_prod_type_cd + , clm_val_sqnc_num + , clm_dgns_cd + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , clm_poa_ind + , dgns_prcdr_icd_ind + , file_name + , file_date + from {{ ref('stg_parta_diagnosis_code') }} + +) + +/* dedupe full rows that may appear in multiple files */ +, add_row_num as ( + + select *, row_number() over ( + partition by + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_prod_type_cd + , clm_val_sqnc_num + , clm_dgns_cd + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , clm_poa_ind + , dgns_prcdr_icd_ind + order by file_date desc + ) as row_num + from staged_data + where bene_mbi_id is not null /* added to prevent dupes during pivot */ + +) + +/* cast data types before pivot operation */ +select + cast(cur_clm_uniq_id as {{ dbt.type_string() }} ) as cur_clm_uniq_id + , cast(bene_mbi_id as {{ dbt.type_string() }} ) as bene_mbi_id + , cast(bene_hic_num as {{ dbt.type_string() }} ) as bene_hic_num + , cast(clm_type_cd as {{ dbt.type_string() }} ) as clm_type_cd + , cast(clm_prod_type_cd as {{ dbt.type_string() }} ) as clm_prod_type_cd + , cast(cast(clm_val_sqnc_num as integer) as {{ dbt.type_string() }} ) as clm_val_sqnc_num + , cast(clm_dgns_cd as {{ dbt.type_string() }} ) as clm_dgns_cd + , cast(bene_eqtbl_bic_hicn_num as {{ dbt.type_string() }} ) as bene_eqtbl_bic_hicn_num + , cast(prvdr_oscar_num as {{ dbt.type_string() }} ) as prvdr_oscar_num + , cast(clm_from_dt as {{ dbt.type_string() }} ) as clm_from_dt + , cast(clm_thru_dt as {{ dbt.type_string() }} ) as clm_thru_dt + , cast(clm_poa_ind as {{ dbt.type_string() }} ) as clm_poa_ind + , cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) as dgns_prcdr_icd_ind + , file_name + , file_date +from add_row_num +where row_num = 1 \ No newline at end of file diff --git a/models/intermediate/diagnosis_pivot.sql b/models/intermediate/int_diagnosis_pivot.sql similarity index 88% rename from models/intermediate/diagnosis_pivot.sql rename to models/intermediate/int_diagnosis_pivot.sql index 4df3ff3a..6096d465 100644 --- a/models/intermediate/diagnosis_pivot.sql +++ b/models/intermediate/int_diagnosis_pivot.sql @@ -1,105 +1,103 @@ -with diagnois_pivot as ( - - select - cur_clm_uniq_id - , bene_mbi_id - , dgns_prcdr_icd_ind - , {{ dbt_utils.pivot( - column='clm_val_sqnc_num' - , values=['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25'] - , agg='max' - , then_value='clm_dgns_cd' - , else_value='null' - , prefix='diagnosis_code_' - , quote_identifiers=false - ) }} - from {{ ref('stg_parta_diagnosis_code') }} - group by - cur_clm_uniq_id - , bene_mbi_id - , dgns_prcdr_icd_ind - -), - -poa_pivot as ( - - select - cur_clm_uniq_id - , bene_mbi_id - , dgns_prcdr_icd_ind - , {{ dbt_utils.pivot( - column='clm_val_sqnc_num' - , values=['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25'] - , agg='max' - , then_value='clm_poa_ind' - , else_value='null' - , prefix='diagnosis_poa_' - , quote_identifiers=false - ) }} - from {{ ref('stg_parta_diagnosis_code') }} - group by - cur_clm_uniq_id - , bene_mbi_id - , dgns_prcdr_icd_ind - -) - -select - dx.cur_clm_uniq_id - , dx.bene_mbi_id - , dx.dgns_prcdr_icd_ind - , dx.diagnosis_code_1 - , dx.diagnosis_code_2 - , dx.diagnosis_code_3 - , dx.diagnosis_code_4 - , dx.diagnosis_code_5 - , dx.diagnosis_code_6 - , dx.diagnosis_code_7 - , dx.diagnosis_code_8 - , dx.diagnosis_code_9 - , dx.diagnosis_code_10 - , dx.diagnosis_code_11 - , dx.diagnosis_code_12 - , dx.diagnosis_code_13 - , dx.diagnosis_code_14 - , dx.diagnosis_code_15 - , dx.diagnosis_code_16 - , dx.diagnosis_code_17 - , dx.diagnosis_code_18 - , dx.diagnosis_code_19 - , dx.diagnosis_code_20 - , dx.diagnosis_code_21 - , dx.diagnosis_code_22 - , dx.diagnosis_code_23 - , dx.diagnosis_code_24 - , dx.diagnosis_code_25 - , poa.diagnosis_poa_1 - , poa.diagnosis_poa_2 - , poa.diagnosis_poa_3 - , poa.diagnosis_poa_4 - , poa.diagnosis_poa_5 - , poa.diagnosis_poa_6 - , poa.diagnosis_poa_7 - , poa.diagnosis_poa_8 - , poa.diagnosis_poa_9 - , poa.diagnosis_poa_10 - , poa.diagnosis_poa_11 - , poa.diagnosis_poa_12 - , poa.diagnosis_poa_13 - , poa.diagnosis_poa_14 - , poa.diagnosis_poa_15 - , poa.diagnosis_poa_16 - , poa.diagnosis_poa_17 - , poa.diagnosis_poa_18 - , poa.diagnosis_poa_19 - , poa.diagnosis_poa_20 - , poa.diagnosis_poa_21 - , poa.diagnosis_poa_22 - , poa.diagnosis_poa_23 - , poa.diagnosis_poa_24 - , poa.diagnosis_poa_25 -from diagnois_pivot as dx - inner join poa_pivot as poa - on dx.cur_clm_uniq_id = poa.cur_clm_uniq_id -/* filtering out null values from seed file */ -where dx.cur_clm_uniq_id is not null \ No newline at end of file +with diagnois_pivot as ( + + select + cur_clm_uniq_id + , bene_mbi_id + , dgns_prcdr_icd_ind + , {{ dbt_utils.pivot( + column='clm_val_sqnc_num' + , values=['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25'] + , agg='max' + , then_value='clm_dgns_cd' + , else_value='null' + , prefix='diagnosis_code_' + , quote_identifiers=false + ) }} + from {{ ref('int_diagnosis_deduped') }} + group by + cur_clm_uniq_id + , bene_mbi_id + , dgns_prcdr_icd_ind + +), + +poa_pivot as ( + + select + cur_clm_uniq_id + , bene_mbi_id + , dgns_prcdr_icd_ind + , {{ dbt_utils.pivot( + column='clm_val_sqnc_num' + , values=['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25'] + , agg='max' + , then_value='clm_poa_ind' + , else_value='null' + , prefix='diagnosis_poa_' + , quote_identifiers=false + ) }} + from {{ ref('int_diagnosis_deduped') }} + group by + cur_clm_uniq_id + , bene_mbi_id + , dgns_prcdr_icd_ind + +) + +select + dx.cur_clm_uniq_id + , dx.bene_mbi_id + , dx.dgns_prcdr_icd_ind + , dx.diagnosis_code_1 + , dx.diagnosis_code_2 + , dx.diagnosis_code_3 + , dx.diagnosis_code_4 + , dx.diagnosis_code_5 + , dx.diagnosis_code_6 + , dx.diagnosis_code_7 + , dx.diagnosis_code_8 + , dx.diagnosis_code_9 + , dx.diagnosis_code_10 + , dx.diagnosis_code_11 + , dx.diagnosis_code_12 + , dx.diagnosis_code_13 + , dx.diagnosis_code_14 + , dx.diagnosis_code_15 + , dx.diagnosis_code_16 + , dx.diagnosis_code_17 + , dx.diagnosis_code_18 + , dx.diagnosis_code_19 + , dx.diagnosis_code_20 + , dx.diagnosis_code_21 + , dx.diagnosis_code_22 + , dx.diagnosis_code_23 + , dx.diagnosis_code_24 + , dx.diagnosis_code_25 + , poa.diagnosis_poa_1 + , poa.diagnosis_poa_2 + , poa.diagnosis_poa_3 + , poa.diagnosis_poa_4 + , poa.diagnosis_poa_5 + , poa.diagnosis_poa_6 + , poa.diagnosis_poa_7 + , poa.diagnosis_poa_8 + , poa.diagnosis_poa_9 + , poa.diagnosis_poa_10 + , poa.diagnosis_poa_11 + , poa.diagnosis_poa_12 + , poa.diagnosis_poa_13 + , poa.diagnosis_poa_14 + , poa.diagnosis_poa_15 + , poa.diagnosis_poa_16 + , poa.diagnosis_poa_17 + , poa.diagnosis_poa_18 + , poa.diagnosis_poa_19 + , poa.diagnosis_poa_20 + , poa.diagnosis_poa_21 + , poa.diagnosis_poa_22 + , poa.diagnosis_poa_23 + , poa.diagnosis_poa_24 + , poa.diagnosis_poa_25 +from diagnois_pivot as dx + inner join poa_pivot as poa + on dx.cur_clm_uniq_id = poa.cur_clm_uniq_id \ No newline at end of file diff --git a/models/intermediate/int_dme_claim_adr.sql b/models/intermediate/int_dme_claim_adr.sql new file mode 100644 index 00000000..181b87a5 --- /dev/null +++ b/models/intermediate/int_dme_claim_adr.sql @@ -0,0 +1,215 @@ +with staged_data as ( + + select + cur_clm_uniq_id + , clm_line_num + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_from_dt + , clm_thru_dt + , clm_fed_type_srvc_cd + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + , clm_prmry_pyr_cd + , payto_prvdr_npi_num + , ordrg_prvdr_npi_num + , clm_carr_pmt_dnl_cd + , clm_prcsg_ind_cd + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_idr_ld_dt + , clm_cntl_num + , bene_eqtbl_bic_hicn_num + , clm_line_alowd_chrg_amt + , clm_disp_cd + , file_name + , file_date + from {{ ref('stg_partb_dme') }} + +) + +, beneficiary_xref as ( + + select * from {{ ref('int_beneficiary_xref_deduped') }} + +) + +/* dedupe full rows that may appear in multiple files */ +, add_row_num as ( + + select *, row_number() over ( + partition by + cur_clm_uniq_id + , clm_line_num + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_from_dt + , clm_thru_dt + , clm_fed_type_srvc_cd + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + , clm_prmry_pyr_cd + , payto_prvdr_npi_num + , ordrg_prvdr_npi_num + , clm_carr_pmt_dnl_cd + , clm_prcsg_ind_cd + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_idr_ld_dt + , clm_cntl_num + , bene_eqtbl_bic_hicn_num + , clm_line_alowd_chrg_amt + , clm_disp_cd + order by file_date desc + ) as row_num + from staged_data + +) + +/* source fields not mapped or used for adjustment logic are commented out */ +, dedupe as ( + + select + cur_clm_uniq_id + , clm_line_num + , bene_mbi_id + /*, bene_hic_num*/ + /*, clm_type_cd*/ + , clm_from_dt + , clm_thru_dt + /*, clm_fed_type_srvc_cd*/ + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + /*, clm_prmry_pyr_cd*/ + , payto_prvdr_npi_num + , ordrg_prvdr_npi_num + /*, clm_carr_pmt_dnl_cd*/ + /*, clm_prcsg_ind_cd*/ + , clm_adjsmt_type_cd + , clm_efctv_dt + /*, clm_idr_ld_dt*/ + , clm_cntl_num + /*, bene_eqtbl_bic_hicn_num*/ + , clm_line_alowd_chrg_amt + /*, clm_disp_cd*/ + , file_name + , file_date + from add_row_num + where row_num = 1 + +) + +/* coalesce current MBI from XREF if exists and MBI on claim */ +, add_current_mbi as ( + + select + dedupe.cur_clm_uniq_id + , dedupe.clm_line_num + , coalesce(beneficiary_xref.crnt_num, dedupe.bene_mbi_id) as current_bene_mbi_id + , dedupe.clm_from_dt + , dedupe.clm_thru_dt + , dedupe.clm_pos_cd + , dedupe.clm_line_from_dt + , dedupe.clm_line_thru_dt + , dedupe.clm_line_hcpcs_cd + , dedupe.clm_line_cvrd_pd_amt + , dedupe.payto_prvdr_npi_num + , dedupe.ordrg_prvdr_npi_num + , dedupe.clm_adjsmt_type_cd + , dedupe.clm_efctv_dt + , dedupe.clm_cntl_num + , dedupe.clm_line_alowd_chrg_amt + , dedupe.file_name + , dedupe.file_date + from dedupe + left join beneficiary_xref + on dedupe.bene_mbi_id = beneficiary_xref.prvs_num + + +) + +/* + 1) apply adjustment logic by grouping part B DME claims by their natural keys: + - CLM_CNTL_NUM + - Most Recent MBI + - CLM_LINE_NUM (not listed in CCLF docs, but used to prevent line detail loss) + + 2) sort grouped claims by the latest CLM_EFCTV_DT and CUR_CLM_UNIQ_ID since CLM_ADJSMT_TYPE_CD + is not used consistently to indciate the latest final version of an adjusted claim. + + 3) change paid amounts to negative for canceled claims + + (CCLF docs ref: 5.3 Calculating Beneficiary-Level Expenditures) +*/ +, sort_adjusted_claims as ( + + select + cur_clm_uniq_id + , clm_line_num + , current_bene_mbi_id + , clm_from_dt + , clm_thru_dt + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , case + when clm_adjsmt_type_cd = '1' then {{ cast_numeric('clm_line_cvrd_pd_amt') }} * -1 + else {{ cast_numeric('clm_line_cvrd_pd_amt') }} + end as clm_line_cvrd_pd_amt + , payto_prvdr_npi_num + , ordrg_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_cntl_num + , case + when clm_adjsmt_type_cd = '1' then {{ cast_numeric('clm_line_alowd_chrg_amt') }} * -1 + else {{ cast_numeric('clm_line_alowd_chrg_amt') }} + end as clm_line_alowd_chrg_amt + , file_name + , file_date + , row_number() over ( + partition by + clm_cntl_num + , clm_line_num + , current_bene_mbi_id + order by + clm_efctv_dt desc + , cur_clm_uniq_id desc + ) as row_num + from add_current_mbi + +) + +select + cur_clm_uniq_id + , clm_line_num + , current_bene_mbi_id + , clm_from_dt + , clm_thru_dt + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + , payto_prvdr_npi_num + , ordrg_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_cntl_num + , clm_line_alowd_chrg_amt + , file_name + , file_date + , row_num +from sort_adjusted_claims \ No newline at end of file diff --git a/models/intermediate/int_dme_claim_deduped.sql b/models/intermediate/int_dme_claim_deduped.sql new file mode 100644 index 00000000..9d14ee2d --- /dev/null +++ b/models/intermediate/int_dme_claim_deduped.sql @@ -0,0 +1,578 @@ +with sort_adjusted_claims as ( + + select + cur_clm_uniq_id + , clm_line_num + , current_bene_mbi_id + , clm_from_dt + , clm_thru_dt + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + , payto_prvdr_npi_num + , ordrg_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_cntl_num + , clm_line_alowd_chrg_amt + , file_name + , file_date + , row_num + from {{ ref('int_dme_claim_adr') }} + +) + +/* + sum the adjusted line amounts + + (CCLF docs ref: 5.3 Calculating Beneficiary-Level Expenditures) +*/ +, line_totals as ( + + select + clm_cntl_num + , clm_line_num + , current_bene_mbi_id + , sum(clm_line_cvrd_pd_amt) as sum_clm_line_cvrd_pd_amt + , sum(clm_line_alowd_chrg_amt) as sum_clm_line_alowd_chrg_amt + from sort_adjusted_claims + group by + clm_cntl_num + , clm_line_num + , current_bene_mbi_id + +) + +/* + apply final adjustment logic by selecting latest version of claim, + removing any remaining claims with a canceled status, and adding line totals. +*/ +, filter_claims as ( + + select + sort_adjusted_claims.cur_clm_uniq_id + , sort_adjusted_claims.clm_line_num + , sort_adjusted_claims.current_bene_mbi_id + , sort_adjusted_claims.clm_from_dt + , sort_adjusted_claims.clm_thru_dt + , sort_adjusted_claims.clm_pos_cd + , sort_adjusted_claims.clm_line_from_dt + , sort_adjusted_claims.clm_line_thru_dt + , sort_adjusted_claims.clm_line_hcpcs_cd + , line_totals.sum_clm_line_cvrd_pd_amt as clm_line_cvrd_pd_amt + , sort_adjusted_claims.payto_prvdr_npi_num + , sort_adjusted_claims.ordrg_prvdr_npi_num + , sort_adjusted_claims.clm_adjsmt_type_cd + , sort_adjusted_claims.clm_efctv_dt + , sort_adjusted_claims.clm_cntl_num + , line_totals.sum_clm_line_alowd_chrg_amt as clm_line_alowd_chrg_amt + , sort_adjusted_claims.file_name + , sort_adjusted_claims.file_date + from sort_adjusted_claims + left join line_totals + on sort_adjusted_claims.clm_cntl_num = line_totals.clm_cntl_num + and sort_adjusted_claims.clm_line_num = line_totals.clm_line_num + and sort_adjusted_claims.current_bene_mbi_id = line_totals.current_bene_mbi_id + where sort_adjusted_claims.row_num = 1 + and sort_adjusted_claims.clm_adjsmt_type_cd <> '1' + +) + +/* + remove claim lines where claim ID+line number not unique + even after adjustments have been applied +*/ +, claim_dupes as ( + + select cur_clm_uniq_id, clm_line_num + from filter_claims + group by cur_clm_uniq_id, clm_line_num + having count(*) > 1 + +) + +, remove_dupes as ( + + select filter_claims.* + from filter_claims + left join claim_dupes + on filter_claims.cur_clm_uniq_id = claim_dupes.cur_clm_uniq_id + and filter_claims.clm_line_num = claim_dupes.clm_line_num + where claim_dupes.cur_clm_uniq_id is null + +) + +, mapping as ( + + select + cur_clm_uniq_id as claim_id + , clm_line_num as claim_line_number + , cast('professional' as {{ dbt.type_string() }} ) as claim_type + , current_bene_mbi_id as patient_id + , current_bene_mbi_id as member_id + , cast('medicare' as {{ dbt.type_string() }} ) as payer + , cast('medicare'as {{ dbt.type_string() }} ) as {{ the_tuva_project.quote_column('plan') }} + , case + when clm_from_dt in ('1000-01-01', '9999-12-31') then null + else clm_from_dt + end as claim_start_date + , case + when clm_thru_dt in ('1000-01-01', '9999-12-31') then null + else clm_thru_dt + end as claim_end_date + , case + when clm_line_from_dt in ('1000-01-01', '9999-12-31') then null + else clm_line_from_dt + end as claim_line_start_date + , case + when clm_line_thru_dt in ('1000-01-01', '9999-12-31') then null + else clm_line_thru_dt + end as claim_line_end_date + , cast(null as date) as admission_date + , cast(null as date) as discharge_date + , cast(null as {{ dbt.type_string() }} ) as admit_source_code + , cast(null as {{ dbt.type_string() }} ) as admit_type_code + , cast(null as {{ dbt.type_string() }} ) as discharge_disposition_code + , clm_pos_cd as place_of_service_code + , cast(null as {{ dbt.type_string() }} ) as bill_type_code + , cast(null as {{ dbt.type_string() }} ) as ms_drg_code + , cast(null as {{ dbt.type_string() }} ) as apr_drg_code + , cast(null as {{ dbt.type_string() }} ) as revenue_center_code + , cast(null as {{ dbt.type_string() }} ) as service_unit_quantity + , clm_line_hcpcs_cd as hcpcs_code + , cast(null as {{ dbt.type_string() }} ) as hcpcs_modifier_1 + , cast(null as {{ dbt.type_string() }} ) as hcpcs_modifier_2 + , cast(null as {{ dbt.type_string() }} ) as hcpcs_modifier_3 + , cast(null as {{ dbt.type_string() }} ) as hcpcs_modifier_4 + , cast(null as {{ dbt.type_string() }} ) as hcpcs_modifier_5 + , ordrg_prvdr_npi_num as rendering_npi + , cast(null as {{ dbt.type_string() }} ) as rendering_tin + , payto_prvdr_npi_num as billing_npi + , cast(null as {{ dbt.type_string() }} ) as billing_tin + , cast(null as {{ dbt.type_string() }} ) as facility_npi + , case + when clm_efctv_dt in ('1000-01-01', '9999-12-31') then null + else clm_efctv_dt + end as paid_date + , clm_line_cvrd_pd_amt as paid_amount + , clm_line_alowd_chrg_amt as allowed_amount + , clm_line_alowd_chrg_amt as charge_amount + , cast(null as {{ dbt.type_string() }} ) as coinsurance_amount + , cast(null as {{ dbt.type_string() }} ) as copayment_amount + , cast(null as {{ dbt.type_string() }} ) as deductible_amount + , cast(null as {{ dbt.type_string() }} ) as total_cost_amount + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_type + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_1 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_2 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_3 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_4 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_5 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_6 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_7 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_8 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_9 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_10 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_11 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_12 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_13 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_14 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_15 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_16 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_17 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_18 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_19 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_20 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_21 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_22 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_23 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_24 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_25 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_1 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_2 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_3 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_4 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_5 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_6 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_7 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_8 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_9 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_10 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_11 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_12 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_13 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_14 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_15 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_16 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_17 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_18 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_19 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_20 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_21 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_22 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_23 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_24 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_25 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_type + , cast(null as {{ dbt.type_string() }} ) as procedure_code_1 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_2 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_3 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_4 + , cast(null as {{ dbt.type_string() }} ) procedure_code_5 + , cast(null as {{ dbt.type_string() }} ) procedure_code_6 + , cast(null as {{ dbt.type_string() }} ) procedure_code_7 + , cast(null as {{ dbt.type_string() }} ) procedure_code_8 + , cast(null as {{ dbt.type_string() }} ) procedure_code_9 + , cast(null as {{ dbt.type_string() }} ) procedure_code_10 + , cast(null as {{ dbt.type_string() }} ) procedure_code_11 + , cast(null as {{ dbt.type_string() }} ) procedure_code_12 + , cast(null as {{ dbt.type_string() }} ) procedure_code_13 + , cast(null as {{ dbt.type_string() }} ) procedure_code_14 + , cast(null as {{ dbt.type_string() }} ) procedure_code_15 + , cast(null as {{ dbt.type_string() }} ) procedure_code_16 + , cast(null as {{ dbt.type_string() }} ) procedure_code_17 + , cast(null as {{ dbt.type_string() }} ) procedure_code_18 + , cast(null as {{ dbt.type_string() }} ) procedure_code_19 + , cast(null as {{ dbt.type_string() }} ) procedure_code_20 + , cast(null as {{ dbt.type_string() }} ) procedure_code_21 + , cast(null as {{ dbt.type_string() }} ) procedure_code_22 + , cast(null as {{ dbt.type_string() }} ) procedure_code_23 + , cast(null as {{ dbt.type_string() }} ) procedure_code_24 + , cast(null as {{ dbt.type_string() }} ) procedure_code_25 + , cast(null as date) as procedure_date_1 + , cast(null as date) as procedure_date_2 + , cast(null as date) as procedure_date_3 + , cast(null as date) as procedure_date_4 + , cast(null as date) as procedure_date_5 + , cast(null as date) as procedure_date_6 + , cast(null as date) as procedure_date_7 + , cast(null as date) as procedure_date_8 + , cast(null as date) as procedure_date_9 + , cast(null as date) as procedure_date_10 + , cast(null as date) as procedure_date_11 + , cast(null as date) as procedure_date_12 + , cast(null as date) as procedure_date_13 + , cast(null as date) as procedure_date_14 + , cast(null as date) as procedure_date_15 + , cast(null as date) as procedure_date_16 + , cast(null as date) as procedure_date_17 + , cast(null as date) as procedure_date_18 + , cast(null as date) as procedure_date_19 + , cast(null as date) as procedure_date_20 + , cast(null as date) as procedure_date_21 + , cast(null as date) as procedure_date_22 + , cast(null as date) as procedure_date_23 + , cast(null as date) as procedure_date_24 + , cast(null as date) as procedure_date_25 + , 1 as in_network_flag + , cast('medicare cclf' as {{ dbt.type_string() }} ) as data_source + , file_name + , file_date as ingest_datetime + from remove_dupes + +) + +, add_data_types as ( + + select + cast(claim_id as {{ dbt.type_string() }} ) as claim_id + , cast(claim_line_number as integer) as claim_line_number + , cast(claim_type as {{ dbt.type_string() }} ) as claim_type + , cast(patient_id as {{ dbt.type_string() }} ) as patient_id + , cast(member_id as {{ dbt.type_string() }} ) as member_id + , cast(payer as {{ dbt.type_string() }} ) as payer + , cast({{ the_tuva_project.quote_column('plan') }} as {{ dbt.type_string() }} ) as {{ the_tuva_project.quote_column('plan') }} + , {{ try_to_cast_date('claim_start_date', 'YYYY-MM-DD') }} as claim_start_date + , {{ try_to_cast_date('claim_end_date', 'YYYY-MM-DD') }} as claim_end_date + , {{ try_to_cast_date('claim_line_start_date', 'YYYY-MM-DD') }} as claim_line_start_date + , {{ try_to_cast_date('claim_line_end_date', 'YYYY-MM-DD') }} as claim_line_end_date + , {{ try_to_cast_date('admission_date', 'YYYY-MM-DD') }} as admission_date + , {{ try_to_cast_date('discharge_date', 'YYYY-MM-DD') }} as discharge_date + , cast(admit_source_code as {{ dbt.type_string() }} ) as admit_source_code + , cast(admit_type_code as {{ dbt.type_string() }} ) as admit_type_code + , cast(discharge_disposition_code as {{ dbt.type_string() }} ) as discharge_disposition_code + , cast(place_of_service_code as {{ dbt.type_string() }} ) as place_of_service_code + , cast(bill_type_code as {{ dbt.type_string() }} ) as bill_type_code + , cast(ms_drg_code as {{ dbt.type_string() }} ) as ms_drg_code + , cast(apr_drg_code as {{ dbt.type_string() }} ) as apr_drg_code + , cast(revenue_center_code as {{ dbt.type_string() }} ) as revenue_center_code + , {{ cast_numeric('service_unit_quantity') }} as service_unit_quantity + , cast(hcpcs_code as {{ dbt.type_string() }} ) as hcpcs_code + , cast(hcpcs_modifier_1 as {{ dbt.type_string() }} ) as hcpcs_modifier_1 + , cast(hcpcs_modifier_2 as {{ dbt.type_string() }} ) as hcpcs_modifier_2 + , cast(hcpcs_modifier_3 as {{ dbt.type_string() }} ) as hcpcs_modifier_3 + , cast(hcpcs_modifier_4 as {{ dbt.type_string() }} ) as hcpcs_modifier_4 + , cast(hcpcs_modifier_5 as {{ dbt.type_string() }} ) as hcpcs_modifier_5 + , cast(rendering_npi as {{ dbt.type_string() }} ) as rendering_npi + , cast(rendering_tin as {{ dbt.type_string() }} ) as rendering_tin + , cast(billing_npi as {{ dbt.type_string() }} ) as billing_npi + , cast(billing_tin as {{ dbt.type_string() }} ) as billing_tin + , cast(facility_npi as {{ dbt.type_string() }} ) as facility_npi + , {{ try_to_cast_date('paid_date', 'YYYY-MM-DD') }} as paid_date + , {{ cast_numeric('paid_amount') }} as paid_amount + , {{ cast_numeric('allowed_amount') }} as allowed_amount + , {{ cast_numeric('charge_amount') }} as charge_amount + , {{ cast_numeric('coinsurance_amount') }} as coinsurance_amount + , {{ cast_numeric('copayment_amount') }} as copayment_amount + , {{ cast_numeric('deductible_amount') }} as deductible_amount + , {{ cast_numeric('total_cost_amount') }} as total_cost_amount + , cast(diagnosis_code_type as {{ dbt.type_string() }} ) as diagnosis_code_type + , cast(diagnosis_code_1 as {{ dbt.type_string() }} ) as diagnosis_code_1 + , cast(diagnosis_code_2 as {{ dbt.type_string() }} ) as diagnosis_code_2 + , cast(diagnosis_code_3 as {{ dbt.type_string() }} ) as diagnosis_code_3 + , cast(diagnosis_code_4 as {{ dbt.type_string() }} ) as diagnosis_code_4 + , cast(diagnosis_code_5 as {{ dbt.type_string() }} ) as diagnosis_code_5 + , cast(diagnosis_code_6 as {{ dbt.type_string() }} ) as diagnosis_code_6 + , cast(diagnosis_code_7 as {{ dbt.type_string() }} ) as diagnosis_code_7 + , cast(diagnosis_code_8 as {{ dbt.type_string() }} ) as diagnosis_code_8 + , cast(diagnosis_code_9 as {{ dbt.type_string() }} ) as diagnosis_code_9 + , cast(diagnosis_code_10 as {{ dbt.type_string() }} ) as diagnosis_code_10 + , cast(diagnosis_code_11 as {{ dbt.type_string() }} ) as diagnosis_code_11 + , cast(diagnosis_code_12 as {{ dbt.type_string() }} ) as diagnosis_code_12 + , cast(diagnosis_code_13 as {{ dbt.type_string() }} ) as diagnosis_code_13 + , cast(diagnosis_code_14 as {{ dbt.type_string() }} ) as diagnosis_code_14 + , cast(diagnosis_code_15 as {{ dbt.type_string() }} ) as diagnosis_code_15 + , cast(diagnosis_code_16 as {{ dbt.type_string() }} ) as diagnosis_code_16 + , cast(diagnosis_code_17 as {{ dbt.type_string() }} ) as diagnosis_code_17 + , cast(diagnosis_code_18 as {{ dbt.type_string() }} ) as diagnosis_code_18 + , cast(diagnosis_code_19 as {{ dbt.type_string() }} ) as diagnosis_code_19 + , cast(diagnosis_code_20 as {{ dbt.type_string() }} ) as diagnosis_code_20 + , cast(diagnosis_code_21 as {{ dbt.type_string() }} ) as diagnosis_code_21 + , cast(diagnosis_code_22 as {{ dbt.type_string() }} ) as diagnosis_code_22 + , cast(diagnosis_code_23 as {{ dbt.type_string() }} ) as diagnosis_code_23 + , cast(diagnosis_code_24 as {{ dbt.type_string() }} ) as diagnosis_code_24 + , cast(diagnosis_code_25 as {{ dbt.type_string() }} ) as diagnosis_code_25 + , cast(diagnosis_poa_1 as {{ dbt.type_string() }} ) as diagnosis_poa_1 + , cast(diagnosis_poa_2 as {{ dbt.type_string() }} ) as diagnosis_poa_2 + , cast(diagnosis_poa_3 as {{ dbt.type_string() }} ) as diagnosis_poa_3 + , cast(diagnosis_poa_4 as {{ dbt.type_string() }} ) as diagnosis_poa_4 + , cast(diagnosis_poa_5 as {{ dbt.type_string() }} ) as diagnosis_poa_5 + , cast(diagnosis_poa_6 as {{ dbt.type_string() }} ) as diagnosis_poa_6 + , cast(diagnosis_poa_7 as {{ dbt.type_string() }} ) as diagnosis_poa_7 + , cast(diagnosis_poa_8 as {{ dbt.type_string() }} ) as diagnosis_poa_8 + , cast(diagnosis_poa_9 as {{ dbt.type_string() }} ) as diagnosis_poa_9 + , cast(diagnosis_poa_10 as {{ dbt.type_string() }} ) as diagnosis_poa_10 + , cast(diagnosis_poa_11 as {{ dbt.type_string() }} ) as diagnosis_poa_11 + , cast(diagnosis_poa_12 as {{ dbt.type_string() }} ) as diagnosis_poa_12 + , cast(diagnosis_poa_13 as {{ dbt.type_string() }} ) as diagnosis_poa_13 + , cast(diagnosis_poa_14 as {{ dbt.type_string() }} ) as diagnosis_poa_14 + , cast(diagnosis_poa_15 as {{ dbt.type_string() }} ) as diagnosis_poa_15 + , cast(diagnosis_poa_16 as {{ dbt.type_string() }} ) as diagnosis_poa_16 + , cast(diagnosis_poa_17 as {{ dbt.type_string() }} ) as diagnosis_poa_17 + , cast(diagnosis_poa_18 as {{ dbt.type_string() }} ) as diagnosis_poa_18 + , cast(diagnosis_poa_19 as {{ dbt.type_string() }} ) as diagnosis_poa_19 + , cast(diagnosis_poa_20 as {{ dbt.type_string() }} ) as diagnosis_poa_20 + , cast(diagnosis_poa_21 as {{ dbt.type_string() }} ) as diagnosis_poa_21 + , cast(diagnosis_poa_22 as {{ dbt.type_string() }} ) as diagnosis_poa_22 + , cast(diagnosis_poa_23 as {{ dbt.type_string() }} ) as diagnosis_poa_23 + , cast(diagnosis_poa_24 as {{ dbt.type_string() }} ) as diagnosis_poa_24 + , cast(diagnosis_poa_25 as {{ dbt.type_string() }} ) as diagnosis_poa_25 + , cast(procedure_code_type as {{ dbt.type_string() }} ) as procedure_code_type + , cast(procedure_code_1 as {{ dbt.type_string() }} ) as procedure_code_1 + , cast(procedure_code_2 as {{ dbt.type_string() }} ) as procedure_code_2 + , cast(procedure_code_3 as {{ dbt.type_string() }} ) as procedure_code_3 + , cast(procedure_code_4 as {{ dbt.type_string() }} ) as procedure_code_4 + , cast(procedure_code_5 as {{ dbt.type_string() }} ) as procedure_code_5 + , cast(procedure_code_6 as {{ dbt.type_string() }} ) as procedure_code_6 + , cast(procedure_code_7 as {{ dbt.type_string() }} ) as procedure_code_7 + , cast(procedure_code_8 as {{ dbt.type_string() }} ) as procedure_code_8 + , cast(procedure_code_9 as {{ dbt.type_string() }} ) as procedure_code_9 + , cast(procedure_code_10 as {{ dbt.type_string() }} ) as procedure_code_10 + , cast(procedure_code_11 as {{ dbt.type_string() }} ) as procedure_code_11 + , cast(procedure_code_12 as {{ dbt.type_string() }} ) as procedure_code_12 + , cast(procedure_code_13 as {{ dbt.type_string() }} ) as procedure_code_13 + , cast(procedure_code_14 as {{ dbt.type_string() }} ) as procedure_code_14 + , cast(procedure_code_15 as {{ dbt.type_string() }} ) as procedure_code_15 + , cast(procedure_code_16 as {{ dbt.type_string() }} ) as procedure_code_16 + , cast(procedure_code_17 as {{ dbt.type_string() }} ) as procedure_code_17 + , cast(procedure_code_18 as {{ dbt.type_string() }} ) as procedure_code_18 + , cast(procedure_code_19 as {{ dbt.type_string() }} ) as procedure_code_19 + , cast(procedure_code_20 as {{ dbt.type_string() }} ) as procedure_code_20 + , cast(procedure_code_21 as {{ dbt.type_string() }} ) as procedure_code_21 + , cast(procedure_code_22 as {{ dbt.type_string() }} ) as procedure_code_22 + , cast(procedure_code_23 as {{ dbt.type_string() }} ) as procedure_code_23 + , cast(procedure_code_24 as {{ dbt.type_string() }} ) as procedure_code_24 + , cast(procedure_code_25 as {{ dbt.type_string() }} ) as procedure_code_25 + , {{ try_to_cast_date('procedure_date_1', 'YYYY-MM-DD') }} as procedure_date_1 + , {{ try_to_cast_date('procedure_date_2', 'YYYY-MM-DD') }} as procedure_date_2 + , {{ try_to_cast_date('procedure_date_3', 'YYYY-MM-DD') }} as procedure_date_3 + , {{ try_to_cast_date('procedure_date_4', 'YYYY-MM-DD') }} as procedure_date_4 + , {{ try_to_cast_date('procedure_date_5', 'YYYY-MM-DD') }} as procedure_date_5 + , {{ try_to_cast_date('procedure_date_6', 'YYYY-MM-DD') }} as procedure_date_6 + , {{ try_to_cast_date('procedure_date_7', 'YYYY-MM-DD') }} as procedure_date_7 + , {{ try_to_cast_date('procedure_date_8', 'YYYY-MM-DD') }} as procedure_date_8 + , {{ try_to_cast_date('procedure_date_9', 'YYYY-MM-DD') }} as procedure_date_9 + , {{ try_to_cast_date('procedure_date_10', 'YYYY-MM-DD') }} as procedure_date_10 + , {{ try_to_cast_date('procedure_date_11', 'YYYY-MM-DD') }} as procedure_date_11 + , {{ try_to_cast_date('procedure_date_12', 'YYYY-MM-DD') }} as procedure_date_12 + , {{ try_to_cast_date('procedure_date_13', 'YYYY-MM-DD') }} as procedure_date_13 + , {{ try_to_cast_date('procedure_date_14', 'YYYY-MM-DD') }} as procedure_date_14 + , {{ try_to_cast_date('procedure_date_15', 'YYYY-MM-DD') }} as procedure_date_15 + , {{ try_to_cast_date('procedure_date_16', 'YYYY-MM-DD') }} as procedure_date_16 + , {{ try_to_cast_date('procedure_date_17', 'YYYY-MM-DD') }} as procedure_date_17 + , {{ try_to_cast_date('procedure_date_18', 'YYYY-MM-DD') }} as procedure_date_18 + , {{ try_to_cast_date('procedure_date_19', 'YYYY-MM-DD') }} as procedure_date_19 + , {{ try_to_cast_date('procedure_date_20', 'YYYY-MM-DD') }} as procedure_date_20 + , {{ try_to_cast_date('procedure_date_21', 'YYYY-MM-DD') }} as procedure_date_21 + , {{ try_to_cast_date('procedure_date_22', 'YYYY-MM-DD') }} as procedure_date_22 + , {{ try_to_cast_date('procedure_date_23', 'YYYY-MM-DD') }} as procedure_date_23 + , {{ try_to_cast_date('procedure_date_24', 'YYYY-MM-DD') }} as procedure_date_24 + , {{ try_to_cast_date('procedure_date_25', 'YYYY-MM-DD') }} as procedure_date_25 + , cast(in_network_flag as integer) as in_network_flag + , cast(data_source as {{ dbt.type_string() }} ) as data_source + , cast(file_name as {{ dbt.type_string() }} ) as file_name + , cast(ingest_datetime as {{ dbt.type_string() }} ) as ingest_datetime + from mapping + +) + +select + claim_id + , claim_line_number + , claim_type + , patient_id + , member_id + , payer + , {{ the_tuva_project.quote_column('plan') }} + , claim_start_date + , claim_end_date + , claim_line_start_date + , claim_line_end_date + , admission_date + , discharge_date + , admit_source_code + , admit_type_code + , discharge_disposition_code + , place_of_service_code + , bill_type_code + , ms_drg_code + , apr_drg_code + , revenue_center_code + , service_unit_quantity + , hcpcs_code + , hcpcs_modifier_1 + , hcpcs_modifier_2 + , hcpcs_modifier_3 + , hcpcs_modifier_4 + , hcpcs_modifier_5 + , rendering_npi + , rendering_tin + , billing_npi + , billing_tin + , facility_npi + , paid_date + , paid_amount + , allowed_amount + , charge_amount + , coinsurance_amount + , copayment_amount + , deductible_amount + , total_cost_amount + , diagnosis_code_type + , diagnosis_code_1 + , diagnosis_code_2 + , diagnosis_code_3 + , diagnosis_code_4 + , diagnosis_code_5 + , diagnosis_code_6 + , diagnosis_code_7 + , diagnosis_code_8 + , diagnosis_code_9 + , diagnosis_code_10 + , diagnosis_code_11 + , diagnosis_code_12 + , diagnosis_code_13 + , diagnosis_code_14 + , diagnosis_code_15 + , diagnosis_code_16 + , diagnosis_code_17 + , diagnosis_code_18 + , diagnosis_code_19 + , diagnosis_code_20 + , diagnosis_code_21 + , diagnosis_code_22 + , diagnosis_code_23 + , diagnosis_code_24 + , diagnosis_code_25 + , diagnosis_poa_1 + , diagnosis_poa_2 + , diagnosis_poa_3 + , diagnosis_poa_4 + , diagnosis_poa_5 + , diagnosis_poa_6 + , diagnosis_poa_7 + , diagnosis_poa_8 + , diagnosis_poa_9 + , diagnosis_poa_10 + , diagnosis_poa_11 + , diagnosis_poa_12 + , diagnosis_poa_13 + , diagnosis_poa_14 + , diagnosis_poa_15 + , diagnosis_poa_16 + , diagnosis_poa_17 + , diagnosis_poa_18 + , diagnosis_poa_19 + , diagnosis_poa_20 + , diagnosis_poa_21 + , diagnosis_poa_22 + , diagnosis_poa_23 + , diagnosis_poa_24 + , diagnosis_poa_25 + , procedure_code_type + , procedure_code_1 + , procedure_code_2 + , procedure_code_3 + , procedure_code_4 + , procedure_code_5 + , procedure_code_6 + , procedure_code_7 + , procedure_code_8 + , procedure_code_9 + , procedure_code_10 + , procedure_code_11 + , procedure_code_12 + , procedure_code_13 + , procedure_code_14 + , procedure_code_15 + , procedure_code_16 + , procedure_code_17 + , procedure_code_18 + , procedure_code_19 + , procedure_code_20 + , procedure_code_21 + , procedure_code_22 + , procedure_code_23 + , procedure_code_24 + , procedure_code_25 + , procedure_date_1 + , procedure_date_2 + , procedure_date_3 + , procedure_date_4 + , procedure_date_5 + , procedure_date_6 + , procedure_date_7 + , procedure_date_8 + , procedure_date_9 + , procedure_date_10 + , procedure_date_11 + , procedure_date_12 + , procedure_date_13 + , procedure_date_14 + , procedure_date_15 + , procedure_date_16 + , procedure_date_17 + , procedure_date_18 + , procedure_date_19 + , procedure_date_20 + , procedure_date_21 + , procedure_date_22 + , procedure_date_23 + , procedure_date_24 + , procedure_date_25 + , in_network_flag + , data_source + , file_name + , ingest_datetime +from add_data_types \ No newline at end of file diff --git a/models/intermediate/int_enrollment.sql b/models/intermediate/int_enrollment.sql new file mode 100644 index 00000000..beb30bce --- /dev/null +++ b/models/intermediate/int_enrollment.sql @@ -0,0 +1,134 @@ +/* + This model contains logic to convert member month grain to enrollment date + spans, if needed. +*/ + +with enrollment as ( + + select + current_bene_mbi_id + , enrollment_start_date + , enrollment_end_date + , bene_member_month + from {{ ref('stg_enrollment') }} + +) + +, beneficiary_xref as ( + + select + crnt_num + , prvs_num + from {{ ref('int_beneficiary_xref_deduped') }} + +) + +{% if var('member_months_enrollment',False) == false -%} + +select + coalesce(beneficiary_xref.crnt_num, enrollment.current_bene_mbi_id) as current_bene_mbi_id + , enrollment_start_date + , enrollment_end_date +from enrollment + left join beneficiary_xref + on enrollment.current_bene_mbi_id = beneficiary_xref.prvs_num + +{% else -%} + +/* begin logic to create enrollment spans from member months */ +, add_row_num as ( + + select + current_bene_mbi_id + , cast(bene_member_month as date) as bene_member_month + , row_number() over ( + partition by current_bene_mbi_id + order by bene_member_month + ) as row_num + from enrollment + +) + +, add_lag_enrollment as ( + + select + current_bene_mbi_id + , bene_member_month + , row_num + , lag(bene_member_month) over ( + partition by current_bene_mbi_id + order by row_num + ) as lag_enrollment + from add_row_num + +) + +, calculate_lag_diff as ( + + select + current_bene_mbi_id + , bene_member_month + , row_num + , lag_enrollment + , {{ datediff('lag_enrollment', 'bene_member_month', 'month') }} as lag_diff + from add_lag_enrollment + +) + +, calculate_gaps as ( + + select + current_bene_mbi_id + , bene_member_month + , row_num + , lag_enrollment + , lag_diff + , case + when lag_diff > 1 then 1 + else 0 + end as gap_flag + from calculate_lag_diff + +) + +, calculate_groups as ( + + select + current_bene_mbi_id + , bene_member_month + , row_num + , lag_enrollment + , lag_diff + , gap_flag + , sum(gap_flag) over ( + partition by current_bene_mbi_id + order by row_num + rows between unbounded preceding and current row + ) as row_group + from calculate_gaps + +) + +, rollup_groups as ( + + select + current_bene_mbi_id + , row_group + , min(bene_member_month) as enrollment_start_date + , {{ last_day('max(bene_member_month)', 'month') }} as enrollment_end_date + from calculate_groups + group by + current_bene_mbi_id + , row_group + +) + +select + coalesce(beneficiary_xref.crnt_num, rollup_groups.current_bene_mbi_id) as current_bene_mbi_id + , enrollment_start_date + , enrollment_end_date +from rollup_groups + left join beneficiary_xref + on rollup_groups.current_bene_mbi_id = beneficiary_xref.prvs_num + +{%- endif %} \ No newline at end of file diff --git a/models/intermediate/int_institutional_claim_adr.sql b/models/intermediate/int_institutional_claim_adr.sql new file mode 100644 index 00000000..b3eb7318 --- /dev/null +++ b/models/intermediate/int_institutional_claim_adr.sql @@ -0,0 +1,290 @@ +with staged_data as ( + + select + cur_clm_uniq_id + , prvdr_oscar_num + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_from_dt + , clm_thru_dt + , clm_bill_fac_type_cd + , clm_bill_clsfctn_cd + , prncpl_dgns_cd + , admtg_dgns_cd + , clm_mdcr_npmt_rsn_cd + , clm_pmt_amt + , clm_nch_prmry_pyr_cd + , prvdr_fac_fips_st_cd + , bene_ptnt_stus_cd + , dgns_drg_cd + , clm_op_srvc_type_cd + , fac_prvdr_npi_num + , oprtg_prvdr_npi_num + , atndg_prvdr_npi_num + , othr_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_idr_ld_dt + , bene_eqtbl_bic_hicn_num + , clm_admsn_type_cd + , clm_admsn_src_cd + , clm_bill_freq_cd + , clm_query_cd + , dgns_prcdr_icd_ind + , clm_mdcr_instnl_tot_chrg_amt + , clm_mdcr_ip_pps_cptl_ime_amt + , clm_oprtnl_ime_amt + , clm_mdcr_ip_pps_dsprprtnt_amt + , clm_hipps_uncompd_care_amt + , clm_oprtnl_dsprprtnt_amt + , clm_blg_prvdr_oscar_num + , clm_blg_prvdr_npi_num + , clm_oprtg_prvdr_npi_num + , clm_atndg_prvdr_npi_num + , clm_othr_prvdr_npi_num + , clm_cntl_num + , clm_org_cntl_num + , clm_cntrctr_num + , file_name + , file_date + from {{ ref('stg_parta_claims_header') }} + +) + +, beneficiary_xref as ( + + select * from {{ ref('int_beneficiary_xref_deduped') }} + +) + +/* + dedupe full rows that may appear in multiple files +*/ +, add_row_num as ( + + select *, row_number() over ( + partition by + cur_clm_uniq_id + , prvdr_oscar_num + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_from_dt + , clm_thru_dt + , clm_bill_fac_type_cd + , clm_bill_clsfctn_cd + , prncpl_dgns_cd + , admtg_dgns_cd + , clm_mdcr_npmt_rsn_cd + , clm_pmt_amt + , clm_nch_prmry_pyr_cd + , prvdr_fac_fips_st_cd + , bene_ptnt_stus_cd + , dgns_drg_cd + , clm_op_srvc_type_cd + , fac_prvdr_npi_num + , oprtg_prvdr_npi_num + , atndg_prvdr_npi_num + , othr_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_idr_ld_dt + , bene_eqtbl_bic_hicn_num + , clm_admsn_type_cd + , clm_admsn_src_cd + , clm_bill_freq_cd + , clm_query_cd + , dgns_prcdr_icd_ind + , clm_mdcr_instnl_tot_chrg_amt + , clm_mdcr_ip_pps_cptl_ime_amt + , clm_oprtnl_ime_amt + , clm_mdcr_ip_pps_dsprprtnt_amt + , clm_hipps_uncompd_care_amt + , clm_oprtnl_dsprprtnt_amt + , clm_blg_prvdr_oscar_num + , clm_blg_prvdr_npi_num + , clm_oprtg_prvdr_npi_num + , clm_atndg_prvdr_npi_num + , clm_othr_prvdr_npi_num + , clm_cntl_num + , clm_org_cntl_num + , clm_cntrctr_num + order by file_date desc + ) as row_num + from staged_data + +) + +/* + source fields not mapped or used for adjustment logic are commented out +*/ +, dedupe as ( + + select + cur_clm_uniq_id as cur_clm_uniq_id + /*, prvdr_oscar_num*/ + , bene_mbi_id + /*, bene_hic_num*/ + /*, clm_type_cd*/ + , clm_from_dt + , clm_thru_dt + , clm_bill_fac_type_cd + , clm_bill_clsfctn_cd + /*, prncpl_dgns_cd*/ + /*, admtg_dgns_cd*/ + /*, clm_mdcr_npmt_rsn_cd*/ + , clm_pmt_amt + /*, clm_nch_prmry_pyr_cd*/ + /*, prvdr_fac_fips_st_cd*/ + , bene_ptnt_stus_cd + , dgns_drg_cd + /*, clm_op_srvc_type_cd*/ + , fac_prvdr_npi_num + /*, oprtg_prvdr_npi_num*/ + , atndg_prvdr_npi_num + /*, othr_prvdr_npi_num*/ + , clm_adjsmt_type_cd + , clm_efctv_dt + /*, clm_idr_ld_dt*/ + /*, bene_eqtbl_bic_hicn_num*/ + , clm_admsn_type_cd + , clm_admsn_src_cd + , clm_bill_freq_cd + /*, clm_query_cd*/ + , dgns_prcdr_icd_ind + , clm_mdcr_instnl_tot_chrg_amt + /*, clm_mdcr_ip_pps_cptl_ime_amt*/ + /*, clm_oprtnl_ime_amt*/ + /*, clm_mdcr_ip_pps_dsprprtnt_amt*/ + /*, clm_hipps_uncompd_care_amt*/ + /*, clm_oprtnl_dsprprtnt_amt*/ + , clm_blg_prvdr_oscar_num + /*, clm_blg_prvdr_npi_num*/ + /*, clm_oprtg_prvdr_npi_num*/ + /*, clm_atndg_prvdr_npi_num*/ + /*, clm_othr_prvdr_npi_num*/ + /*, clm_cntl_num*/ + /*, clm_org_cntl_num*/ + /*, clm_cntrctr_num*/ + , file_name + , file_date + from add_row_num + where row_num = 1 + +) + +/* coalesce current MBI from XREF if exists and MBI on claim */ +, add_current_mbi as ( + + select + dedupe.cur_clm_uniq_id + , coalesce(beneficiary_xref.crnt_num, dedupe.bene_mbi_id) as current_bene_mbi_id + , dedupe.clm_from_dt + , dedupe.clm_thru_dt + , dedupe.clm_bill_fac_type_cd + , dedupe.clm_bill_clsfctn_cd + , dedupe.clm_pmt_amt + , dedupe.bene_ptnt_stus_cd + , dedupe.dgns_drg_cd + , dedupe.fac_prvdr_npi_num + , dedupe.atndg_prvdr_npi_num + , dedupe.clm_adjsmt_type_cd + , dedupe.clm_efctv_dt + , dedupe.clm_admsn_type_cd + , dedupe.clm_admsn_src_cd + , dedupe.clm_bill_freq_cd + , dedupe.dgns_prcdr_icd_ind + , dedupe.clm_mdcr_instnl_tot_chrg_amt + , dedupe.clm_blg_prvdr_oscar_num + , dedupe.file_name + , dedupe.file_date + from dedupe + left join beneficiary_xref + on dedupe.bene_mbi_id = beneficiary_xref.prvs_num + + +) + +/* + 1) apply adjustment logic by grouping part A claims by their natural keys: + - CLM_BLG_PRVDR_OSCAR_NUM + - CLM_FROM_DT + - CLM_THRU_DT + - Most Recent MBI + + 2) sort grouped claims by the latest CLM_EFCTV_DT and CUR_CLM_UNIQ_ID since CLM_ADJSMT_TYPE_CD + is not used consistently to indicate the latest final version of an adjusted claim. + + 3) change paid amounts to negative for canceled claims + + (CCLF docs ref: 5.3 Calculating Beneficiary-Level Expenditures) +*/ +, sort_adjusted_claims as ( + + select + cur_clm_uniq_id + , current_bene_mbi_id + , clm_from_dt + , clm_thru_dt + , clm_bill_fac_type_cd + , clm_bill_clsfctn_cd + , case + when clm_adjsmt_type_cd = '1' then {{ cast_numeric('clm_pmt_amt') }} * -1 + else {{ cast_numeric('clm_pmt_amt') }} + end as clm_pmt_amt + , bene_ptnt_stus_cd + , dgns_drg_cd + , fac_prvdr_npi_num + , atndg_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_admsn_type_cd + , clm_admsn_src_cd + , clm_bill_freq_cd + , dgns_prcdr_icd_ind + , case + when clm_adjsmt_type_cd = '1' then {{ cast_numeric('clm_mdcr_instnl_tot_chrg_amt') }} * -1 + else {{ cast_numeric('clm_mdcr_instnl_tot_chrg_amt') }} + end as clm_mdcr_instnl_tot_chrg_amt + , clm_blg_prvdr_oscar_num + , file_name + , file_date + , row_number() over ( + partition by + clm_blg_prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , current_bene_mbi_id + order by + clm_efctv_dt desc + , cur_clm_uniq_id desc + ) as row_num + from add_current_mbi + +) + +select + cur_clm_uniq_id + , current_bene_mbi_id + , clm_from_dt + , clm_thru_dt + , clm_bill_fac_type_cd + , clm_bill_clsfctn_cd + , clm_pmt_amt + , bene_ptnt_stus_cd + , dgns_drg_cd + , fac_prvdr_npi_num + , atndg_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_admsn_type_cd + , clm_admsn_src_cd + , clm_bill_freq_cd + , dgns_prcdr_icd_ind + , clm_mdcr_instnl_tot_chrg_amt + , clm_blg_prvdr_oscar_num + , file_name + , file_date + , row_num +from sort_adjusted_claims \ No newline at end of file diff --git a/models/intermediate/int_institutional_claim_deduped.sql b/models/intermediate/int_institutional_claim_deduped.sql new file mode 100644 index 00000000..fa47b89a --- /dev/null +++ b/models/intermediate/int_institutional_claim_deduped.sql @@ -0,0 +1,844 @@ +with sort_adjusted_claims as ( + + select + cur_clm_uniq_id + , current_bene_mbi_id + , clm_from_dt + , clm_thru_dt + , clm_bill_fac_type_cd + , clm_bill_clsfctn_cd + , clm_pmt_amt + , bene_ptnt_stus_cd + , dgns_drg_cd + , fac_prvdr_npi_num + , atndg_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_admsn_type_cd + , clm_admsn_src_cd + , clm_bill_freq_cd + , dgns_prcdr_icd_ind + , clm_mdcr_instnl_tot_chrg_amt + , clm_blg_prvdr_oscar_num + , file_name + , file_date + , row_num + from {{ ref('int_institutional_claim_adr') }} + +) + +, diagnosis_pivot as ( + + select * from {{ ref('int_diagnosis_pivot') }} + +) + +, procedure_pivot as ( + + select * from {{ ref('int_procedure_pivot') }} + +) + +, revenue_center as ( + + select * from {{ ref('int_revenue_center_deduped') }} + +) + +/* + sum the adjusted header amounts + + (CCLF docs ref: 5.3 Calculating Beneficiary-Level Expenditures) +*/ +, header_totals as ( + + select + clm_blg_prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , current_bene_mbi_id + , sum(clm_pmt_amt) as sum_clm_pmt_amt + , sum(clm_mdcr_instnl_tot_chrg_amt) as sum_clm_mdcr_instnl_tot_chrg_amt + from sort_adjusted_claims + group by + clm_blg_prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , current_bene_mbi_id + +) + +/* + apply final adjustment logic by selecting latest version of claim, + removing any remaining claims with a canceled status, and adding header totals. +*/ +, filter_claims as ( + + select + sort_adjusted_claims.cur_clm_uniq_id + , sort_adjusted_claims.current_bene_mbi_id + , sort_adjusted_claims.clm_from_dt + , sort_adjusted_claims.clm_thru_dt + , sort_adjusted_claims.clm_bill_fac_type_cd + , sort_adjusted_claims.clm_bill_clsfctn_cd + , header_totals.sum_clm_pmt_amt as clm_pmt_amt + , sort_adjusted_claims.bene_ptnt_stus_cd + , sort_adjusted_claims.dgns_drg_cd + , sort_adjusted_claims.fac_prvdr_npi_num + , sort_adjusted_claims.atndg_prvdr_npi_num + , sort_adjusted_claims.clm_adjsmt_type_cd + , sort_adjusted_claims.clm_efctv_dt + , sort_adjusted_claims.clm_admsn_type_cd + , sort_adjusted_claims.clm_admsn_src_cd + , sort_adjusted_claims.clm_bill_freq_cd + , sort_adjusted_claims.dgns_prcdr_icd_ind + , header_totals.sum_clm_mdcr_instnl_tot_chrg_amt as clm_mdcr_instnl_tot_chrg_amt + , sort_adjusted_claims.clm_blg_prvdr_oscar_num + , sort_adjusted_claims.file_name + , sort_adjusted_claims.file_date + from sort_adjusted_claims + left join header_totals + on sort_adjusted_claims.clm_blg_prvdr_oscar_num = header_totals.clm_blg_prvdr_oscar_num + and sort_adjusted_claims.clm_from_dt = header_totals.clm_from_dt + and sort_adjusted_claims.clm_thru_dt = header_totals.clm_thru_dt + and sort_adjusted_claims.current_bene_mbi_id = header_totals.current_bene_mbi_id + where sort_adjusted_claims.row_num = 1 + and sort_adjusted_claims.clm_adjsmt_type_cd <> '1' + +) + +, add_claim_line_details as ( + + select + filter_claims.cur_clm_uniq_id + , filter_claims.current_bene_mbi_id + , filter_claims.clm_from_dt + , filter_claims.clm_thru_dt + , filter_claims.clm_bill_fac_type_cd + , filter_claims.clm_bill_clsfctn_cd + , filter_claims.clm_pmt_amt + , filter_claims.bene_ptnt_stus_cd + , filter_claims.dgns_drg_cd + , filter_claims.fac_prvdr_npi_num + , filter_claims.atndg_prvdr_npi_num + , filter_claims.clm_adjsmt_type_cd + , filter_claims.clm_efctv_dt + , filter_claims.clm_admsn_type_cd + , filter_claims.clm_admsn_src_cd + , filter_claims.clm_bill_freq_cd + , filter_claims.dgns_prcdr_icd_ind + , filter_claims.clm_mdcr_instnl_tot_chrg_amt + , filter_claims.clm_blg_prvdr_oscar_num + , filter_claims.file_name + , filter_claims.file_date + , revenue_center.clm_line_num + , revenue_center.clm_line_from_dt + , revenue_center.clm_line_thru_dt + , revenue_center.clm_line_prod_rev_ctr_cd + , revenue_center.clm_line_hcpcs_cd + , revenue_center.clm_line_srvc_unit_qty + , revenue_center.clm_line_cvrd_pd_amt + , revenue_center.hcpcs_1_mdfr_cd + , revenue_center.hcpcs_2_mdfr_cd + , revenue_center.hcpcs_3_mdfr_cd + , revenue_center.hcpcs_4_mdfr_cd + , revenue_center.hcpcs_5_mdfr_cd + , diagnosis_pivot.dgns_prcdr_icd_ind as diagnosis_icd_ind + , diagnosis_pivot.diagnosis_code_1 + , diagnosis_pivot.diagnosis_code_2 + , diagnosis_pivot.diagnosis_code_3 + , diagnosis_pivot.diagnosis_code_4 + , diagnosis_pivot.diagnosis_code_5 + , diagnosis_pivot.diagnosis_code_6 + , diagnosis_pivot.diagnosis_code_7 + , diagnosis_pivot.diagnosis_code_8 + , diagnosis_pivot.diagnosis_code_9 + , diagnosis_pivot.diagnosis_code_10 + , diagnosis_pivot.diagnosis_code_11 + , diagnosis_pivot.diagnosis_code_12 + , diagnosis_pivot.diagnosis_code_13 + , diagnosis_pivot.diagnosis_code_14 + , diagnosis_pivot.diagnosis_code_15 + , diagnosis_pivot.diagnosis_code_16 + , diagnosis_pivot.diagnosis_code_17 + , diagnosis_pivot.diagnosis_code_18 + , diagnosis_pivot.diagnosis_code_19 + , diagnosis_pivot.diagnosis_code_20 + , diagnosis_pivot.diagnosis_code_21 + , diagnosis_pivot.diagnosis_code_22 + , diagnosis_pivot.diagnosis_code_23 + , diagnosis_pivot.diagnosis_code_24 + , diagnosis_pivot.diagnosis_code_25 + , diagnosis_pivot.diagnosis_poa_1 + , diagnosis_pivot.diagnosis_poa_2 + , diagnosis_pivot.diagnosis_poa_3 + , diagnosis_pivot.diagnosis_poa_4 + , diagnosis_pivot.diagnosis_poa_5 + , diagnosis_pivot.diagnosis_poa_6 + , diagnosis_pivot.diagnosis_poa_7 + , diagnosis_pivot.diagnosis_poa_8 + , diagnosis_pivot.diagnosis_poa_9 + , diagnosis_pivot.diagnosis_poa_10 + , diagnosis_pivot.diagnosis_poa_11 + , diagnosis_pivot.diagnosis_poa_12 + , diagnosis_pivot.diagnosis_poa_13 + , diagnosis_pivot.diagnosis_poa_14 + , diagnosis_pivot.diagnosis_poa_15 + , diagnosis_pivot.diagnosis_poa_16 + , diagnosis_pivot.diagnosis_poa_17 + , diagnosis_pivot.diagnosis_poa_18 + , diagnosis_pivot.diagnosis_poa_19 + , diagnosis_pivot.diagnosis_poa_20 + , diagnosis_pivot.diagnosis_poa_21 + , diagnosis_pivot.diagnosis_poa_22 + , diagnosis_pivot.diagnosis_poa_23 + , diagnosis_pivot.diagnosis_poa_24 + , diagnosis_pivot.diagnosis_poa_25 + , procedure_pivot.dgns_prcdr_icd_ind as procedure_icd_ind + , procedure_pivot.procedure_code_1 + , procedure_pivot.procedure_code_2 + , procedure_pivot.procedure_code_3 + , procedure_pivot.procedure_code_4 + , procedure_pivot.procedure_code_5 + , procedure_pivot.procedure_code_6 + , procedure_pivot.procedure_code_7 + , procedure_pivot.procedure_code_8 + , procedure_pivot.procedure_code_9 + , procedure_pivot.procedure_code_10 + , procedure_pivot.procedure_code_11 + , procedure_pivot.procedure_code_12 + , procedure_pivot.procedure_code_13 + , procedure_pivot.procedure_code_14 + , procedure_pivot.procedure_code_15 + , procedure_pivot.procedure_code_16 + , procedure_pivot.procedure_code_17 + , procedure_pivot.procedure_code_18 + , procedure_pivot.procedure_code_19 + , procedure_pivot.procedure_code_20 + , procedure_pivot.procedure_code_21 + , procedure_pivot.procedure_code_22 + , procedure_pivot.procedure_code_23 + , procedure_pivot.procedure_code_24 + , procedure_pivot.procedure_code_25 + , procedure_pivot.procedure_date_1 + , procedure_pivot.procedure_date_2 + , procedure_pivot.procedure_date_3 + , procedure_pivot.procedure_date_4 + , procedure_pivot.procedure_date_5 + , procedure_pivot.procedure_date_6 + , procedure_pivot.procedure_date_7 + , procedure_pivot.procedure_date_8 + , procedure_pivot.procedure_date_9 + , procedure_pivot.procedure_date_10 + , procedure_pivot.procedure_date_11 + , procedure_pivot.procedure_date_12 + , procedure_pivot.procedure_date_13 + , procedure_pivot.procedure_date_14 + , procedure_pivot.procedure_date_15 + , procedure_pivot.procedure_date_16 + , procedure_pivot.procedure_date_17 + , procedure_pivot.procedure_date_18 + , procedure_pivot.procedure_date_19 + , procedure_pivot.procedure_date_20 + , procedure_pivot.procedure_date_21 + , procedure_pivot.procedure_date_22 + , procedure_pivot.procedure_date_23 + , procedure_pivot.procedure_date_24 + , procedure_pivot.procedure_date_25 + from filter_claims + left join revenue_center + on filter_claims.cur_clm_uniq_id = revenue_center.cur_clm_uniq_id + /* adding part a natural keys to prevent duplicate lines */ + and filter_claims.clm_blg_prvdr_oscar_num = prvdr_oscar_num + and filter_claims.clm_from_dt = revenue_center.clm_from_dt + and filter_claims.clm_thru_dt = revenue_center.clm_thru_dt + and filter_claims.current_bene_mbi_id = revenue_center.current_bene_mbi_id + left join diagnosis_pivot + on filter_claims.cur_clm_uniq_id = diagnosis_pivot.cur_clm_uniq_id + and filter_claims.current_bene_mbi_id = diagnosis_pivot.bene_mbi_id + left join procedure_pivot + on filter_claims.cur_clm_uniq_id = procedure_pivot.cur_clm_uniq_id + and filter_claims.current_bene_mbi_id = procedure_pivot.bene_mbi_id + +) + +/* + remove claim lines where claim ID+line number not unique + even after adjustments have been applied +*/ +, claim_dupes as ( + + select cur_clm_uniq_id, clm_line_num + from add_claim_line_details + group by cur_clm_uniq_id, clm_line_num + having count(*) > 1 + +) + +, remove_dupes as ( + + select add_claim_line_details.* + from add_claim_line_details + left join claim_dupes + on add_claim_line_details.cur_clm_uniq_id = claim_dupes.cur_clm_uniq_id + and add_claim_line_details.clm_line_num = claim_dupes.clm_line_num + where claim_dupes.cur_clm_uniq_id is null + +) + +/*** + + Determine claim paid amount following guidance from CCLF docs: + + 3.5 Part A Header Expenditures vs Part A Revenue Center Expenditures + Both the Part A Header file (CCLF1) and the Part A Revenue Center file (CCLF2) contain a + payment related field, entitled CLM_PMT_AMT and CLM_LINE_CVRD_PD_AMT, respectively. + The revenue center payment amounts should only be relied on if they sum to the header level + payment amount. If the revenue center level payment amounts do not sum to the header level + payment amount, then the revenue center level payment amounts should be ignored. + +***/ + +/* get claim payment from original claim header grain */ +, claim_header_total as ( + + select + cur_clm_uniq_id + , {{ cast_numeric('clm_pmt_amt') }} as clm_pmt_amt + from filter_claims + +) + +/* sum claim line totals from deduped claim lines */ +, claim_line_total as ( + + select + cur_clm_uniq_id + , sum({{ cast_numeric('clm_line_cvrd_pd_amt') }}) as sum_clm_line_cvrd_pd_amt + from remove_dupes + group by cur_clm_uniq_id + +) + +/* create flag to determine if claim line payments should be used */ +, check_payment_totals as ( + + select + claim_header_total.cur_clm_uniq_id + , claim_header_total.clm_pmt_amt + , claim_line_total.sum_clm_line_cvrd_pd_amt + , case + when claim_header_total.clm_pmt_amt = claim_line_total.sum_clm_line_cvrd_pd_amt then 1 + else 0 + end as use_line_payments_flag + from claim_header_total + left join claim_line_total + on claim_header_total.cur_clm_uniq_id = claim_line_total.cur_clm_uniq_id + +) + +, mapping as ( + + select + remove_dupes.cur_clm_uniq_id as claim_id + /* fill in line number for claims with no revenue center details */ + , coalesce(cast(clm_line_num as integer), 1) as claim_line_number + , cast('institutional' as {{ dbt.type_string() }} ) as claim_type + , current_bene_mbi_id as patient_id + , current_bene_mbi_id as member_id + , cast('medicare' as {{ dbt.type_string() }} ) as payer + , cast('medicare'as {{ dbt.type_string() }} ) as {{ the_tuva_project.quote_column('plan') }} + , case + when clm_from_dt in ('1000-01-01', '9999-12-31') then null + else clm_from_dt + end as claim_start_date + , case + when clm_thru_dt in ('1000-01-01', '9999-12-31') then null + else clm_thru_dt + end as claim_end_date + , case + when clm_line_from_dt in ('1000-01-01', '9999-12-31') then null + else clm_line_from_dt + end as claim_line_start_date + , case + when clm_line_thru_dt in ('1000-01-01', '9999-12-31') then null + else clm_line_thru_dt + end as claim_line_end_date + , case + when clm_admsn_type_cd is not null then cast(clm_from_dt as date) + else cast(null as date) + end as admission_date + , case + when clm_admsn_type_cd is not null then cast(clm_thru_dt as date) + else cast(null as date) + end as discharge_date + , clm_admsn_src_cd as admit_source_code + , clm_admsn_type_cd as admit_type_code + , bene_ptnt_stus_cd as discharge_disposition_code + , cast(null as {{ dbt.type_string() }} ) as place_of_service_code + , {{ dbt.concat( + [ + "clm_bill_fac_type_cd", + "clm_bill_clsfctn_cd", + "clm_bill_freq_cd" + ] + ) }} as bill_type_code + , dgns_drg_cd as ms_drg_code + , cast(null as {{ dbt.type_string() }} ) as apr_drg_code + , clm_line_prod_rev_ctr_cd as revenue_center_code + , clm_line_srvc_unit_qty as service_unit_quantity + , clm_line_hcpcs_cd as hcpcs_code + , hcpcs_1_mdfr_cd as hcpcs_modifier_1 + , hcpcs_2_mdfr_cd as hcpcs_modifier_2 + , hcpcs_3_mdfr_cd as hcpcs_modifier_3 + , hcpcs_4_mdfr_cd as hcpcs_modifier_4 + , hcpcs_5_mdfr_cd as hcpcs_modifier_5 + , atndg_prvdr_npi_num as rendering_npi + , cast(null as {{ dbt.type_string() }} ) as rendering_tin + , cast(null as {{ dbt.type_string() }} ) as billing_npi + , cast(null as {{ dbt.type_string() }} ) as billing_tin + , fac_prvdr_npi_num as facility_npi + , case + when clm_efctv_dt in ('1000-01-01', '9999-12-31') then null + else clm_efctv_dt + end as paid_date + /* use flag to determine if claim line payments should be used */ + , case + when use_line_payments_flag = 1 then cast(clm_line_cvrd_pd_amt as {{ dbt.type_string() }} ) + when use_line_payments_flag = 0 and coalesce(cast(clm_line_num as integer), 1) = 1 + then cast(remove_dupes.clm_pmt_amt as {{ dbt.type_string() }} ) + else cast(0 as {{ dbt.type_string() }} ) + end as paid_amount + , cast(null as {{ dbt.type_string() }} ) as allowed_amount + , case + when clm_line_prod_rev_ctr_cd = '0001' + then cast(clm_mdcr_instnl_tot_chrg_amt as {{ dbt.type_string() }} ) + else cast(null as {{ dbt.type_string() }} ) + end as charge_amount + , cast(null as {{ dbt.type_string() }} ) as coinsurance_amount + , cast(null as {{ dbt.type_string() }} ) as copayment_amount + , cast(null as {{ dbt.type_string() }} ) as deductible_amount + , cast(null as {{ dbt.type_string() }} ) as total_cost_amount + , case + when cast(diagnosis_icd_ind as {{ dbt.type_string() }} ) = '0' then 'icd-10-cm' + when cast(diagnosis_icd_ind as {{ dbt.type_string() }} ) = '9' then 'icd-9-cm' + else cast(diagnosis_icd_ind as {{ dbt.type_string() }} ) + end as diagnosis_code_type + , diagnosis_code_1 + , diagnosis_code_2 + , diagnosis_code_3 + , diagnosis_code_4 + , diagnosis_code_5 + , diagnosis_code_6 + , diagnosis_code_7 + , diagnosis_code_8 + , diagnosis_code_9 + , diagnosis_code_10 + , diagnosis_code_11 + , diagnosis_code_12 + , diagnosis_code_13 + , diagnosis_code_14 + , diagnosis_code_15 + , diagnosis_code_16 + , diagnosis_code_17 + , diagnosis_code_18 + , diagnosis_code_19 + , diagnosis_code_20 + , diagnosis_code_21 + , diagnosis_code_22 + , diagnosis_code_23 + , diagnosis_code_24 + , diagnosis_code_25 + , diagnosis_poa_1 + , diagnosis_poa_2 + , diagnosis_poa_3 + , diagnosis_poa_4 + , diagnosis_poa_5 + , diagnosis_poa_6 + , diagnosis_poa_7 + , diagnosis_poa_8 + , diagnosis_poa_9 + , diagnosis_poa_10 + , diagnosis_poa_11 + , diagnosis_poa_12 + , diagnosis_poa_13 + , diagnosis_poa_14 + , diagnosis_poa_15 + , diagnosis_poa_16 + , diagnosis_poa_17 + , diagnosis_poa_18 + , diagnosis_poa_19 + , diagnosis_poa_20 + , diagnosis_poa_21 + , diagnosis_poa_22 + , diagnosis_poa_23 + , diagnosis_poa_24 + , diagnosis_poa_25 + , case + when cast(procedure_icd_ind as {{ dbt.type_string() }} ) = '0' then 'icd-10-cm' + when cast(procedure_icd_ind as {{ dbt.type_string() }} ) = '9' then 'icd-9-cm' + else cast(procedure_icd_ind as {{ dbt.type_string() }} ) + end as procedure_code_type + , procedure_code_1 + , procedure_code_2 + , procedure_code_3 + , procedure_code_4 + , procedure_code_5 + , procedure_code_6 + , procedure_code_7 + , procedure_code_8 + , procedure_code_9 + , procedure_code_10 + , procedure_code_11 + , procedure_code_12 + , procedure_code_13 + , procedure_code_14 + , procedure_code_15 + , procedure_code_16 + , procedure_code_17 + , procedure_code_18 + , procedure_code_19 + , procedure_code_20 + , procedure_code_21 + , procedure_code_22 + , procedure_code_23 + , procedure_code_24 + , procedure_code_25 + , procedure_date_1 + , procedure_date_2 + , procedure_date_3 + , procedure_date_4 + , procedure_date_5 + , procedure_date_6 + , procedure_date_7 + , procedure_date_8 + , procedure_date_9 + , procedure_date_10 + , procedure_date_11 + , procedure_date_12 + , procedure_date_13 + , procedure_date_14 + , procedure_date_15 + , procedure_date_16 + , procedure_date_17 + , procedure_date_18 + , procedure_date_19 + , procedure_date_20 + , procedure_date_21 + , procedure_date_22 + , procedure_date_23 + , procedure_date_24 + , procedure_date_25 + , 1 as in_network_flag + , cast('medicare cclf' as {{ dbt.type_string() }} ) as data_source + , file_name + , file_date as ingest_datetime + from remove_dupes + left join check_payment_totals + on remove_dupes.cur_clm_uniq_id = check_payment_totals.cur_clm_uniq_id + +) + +, add_data_types as ( + + select + cast(claim_id as {{ dbt.type_string() }} ) as claim_id + , cast(claim_line_number as integer) as claim_line_number + , cast(claim_type as {{ dbt.type_string() }} ) as claim_type + , cast(patient_id as {{ dbt.type_string() }} ) as patient_id + , cast(member_id as {{ dbt.type_string() }} ) as member_id + , cast(payer as {{ dbt.type_string() }} ) as payer + , cast({{ the_tuva_project.quote_column('plan') }} as {{ dbt.type_string() }} ) as {{ the_tuva_project.quote_column('plan') }} + , {{ try_to_cast_date('claim_start_date', 'YYYY-MM-DD') }} as claim_start_date + , {{ try_to_cast_date('claim_end_date', 'YYYY-MM-DD') }} as claim_end_date + , {{ try_to_cast_date('claim_line_start_date', 'YYYY-MM-DD') }} as claim_line_start_date + , {{ try_to_cast_date('claim_line_end_date', 'YYYY-MM-DD') }} as claim_line_end_date + , {{ try_to_cast_date('admission_date', 'YYYY-MM-DD') }} as admission_date + , {{ try_to_cast_date('discharge_date', 'YYYY-MM-DD') }} as discharge_date + , cast(admit_source_code as {{ dbt.type_string() }} ) as admit_source_code + , cast(admit_type_code as {{ dbt.type_string() }} ) as admit_type_code + , cast(discharge_disposition_code as {{ dbt.type_string() }} ) as discharge_disposition_code + , cast(place_of_service_code as {{ dbt.type_string() }} ) as place_of_service_code + , cast(bill_type_code as {{ dbt.type_string() }} ) as bill_type_code + , cast(ms_drg_code as {{ dbt.type_string() }} ) as ms_drg_code + , cast(apr_drg_code as {{ dbt.type_string() }} ) as apr_drg_code + , cast(revenue_center_code as {{ dbt.type_string() }} ) as revenue_center_code + , {{ cast_numeric('service_unit_quantity') }} as service_unit_quantity + , cast(hcpcs_code as {{ dbt.type_string() }} ) as hcpcs_code + , cast(hcpcs_modifier_1 as {{ dbt.type_string() }} ) as hcpcs_modifier_1 + , cast(hcpcs_modifier_2 as {{ dbt.type_string() }} ) as hcpcs_modifier_2 + , cast(hcpcs_modifier_3 as {{ dbt.type_string() }} ) as hcpcs_modifier_3 + , cast(hcpcs_modifier_4 as {{ dbt.type_string() }} ) as hcpcs_modifier_4 + , cast(hcpcs_modifier_5 as {{ dbt.type_string() }} ) as hcpcs_modifier_5 + , cast(rendering_npi as {{ dbt.type_string() }} ) as rendering_npi + , cast(rendering_tin as {{ dbt.type_string() }} ) as rendering_tin + , cast(billing_npi as {{ dbt.type_string() }} ) as billing_npi + , cast(billing_tin as {{ dbt.type_string() }} ) as billing_tin + , cast(facility_npi as {{ dbt.type_string() }} ) as facility_npi + , {{ try_to_cast_date('paid_date', 'YYYY-MM-DD') }} as paid_date + , {{ cast_numeric('paid_amount') }} as paid_amount + , {{ cast_numeric('allowed_amount') }} as allowed_amount + , {{ cast_numeric('charge_amount') }} as charge_amount + , {{ cast_numeric('coinsurance_amount') }} as coinsurance_amount + , {{ cast_numeric('copayment_amount') }} as copayment_amount + , {{ cast_numeric('deductible_amount') }} as deductible_amount + , {{ cast_numeric('total_cost_amount') }} as total_cost_amount + , cast(diagnosis_code_type as {{ dbt.type_string() }} ) as diagnosis_code_type + , cast(diagnosis_code_1 as {{ dbt.type_string() }} ) as diagnosis_code_1 + , cast(diagnosis_code_2 as {{ dbt.type_string() }} ) as diagnosis_code_2 + , cast(diagnosis_code_3 as {{ dbt.type_string() }} ) as diagnosis_code_3 + , cast(diagnosis_code_4 as {{ dbt.type_string() }} ) as diagnosis_code_4 + , cast(diagnosis_code_5 as {{ dbt.type_string() }} ) as diagnosis_code_5 + , cast(diagnosis_code_6 as {{ dbt.type_string() }} ) as diagnosis_code_6 + , cast(diagnosis_code_7 as {{ dbt.type_string() }} ) as diagnosis_code_7 + , cast(diagnosis_code_8 as {{ dbt.type_string() }} ) as diagnosis_code_8 + , cast(diagnosis_code_9 as {{ dbt.type_string() }} ) as diagnosis_code_9 + , cast(diagnosis_code_10 as {{ dbt.type_string() }} ) as diagnosis_code_10 + , cast(diagnosis_code_11 as {{ dbt.type_string() }} ) as diagnosis_code_11 + , cast(diagnosis_code_12 as {{ dbt.type_string() }} ) as diagnosis_code_12 + , cast(diagnosis_code_13 as {{ dbt.type_string() }} ) as diagnosis_code_13 + , cast(diagnosis_code_14 as {{ dbt.type_string() }} ) as diagnosis_code_14 + , cast(diagnosis_code_15 as {{ dbt.type_string() }} ) as diagnosis_code_15 + , cast(diagnosis_code_16 as {{ dbt.type_string() }} ) as diagnosis_code_16 + , cast(diagnosis_code_17 as {{ dbt.type_string() }} ) as diagnosis_code_17 + , cast(diagnosis_code_18 as {{ dbt.type_string() }} ) as diagnosis_code_18 + , cast(diagnosis_code_19 as {{ dbt.type_string() }} ) as diagnosis_code_19 + , cast(diagnosis_code_20 as {{ dbt.type_string() }} ) as diagnosis_code_20 + , cast(diagnosis_code_21 as {{ dbt.type_string() }} ) as diagnosis_code_21 + , cast(diagnosis_code_22 as {{ dbt.type_string() }} ) as diagnosis_code_22 + , cast(diagnosis_code_23 as {{ dbt.type_string() }} ) as diagnosis_code_23 + , cast(diagnosis_code_24 as {{ dbt.type_string() }} ) as diagnosis_code_24 + , cast(diagnosis_code_25 as {{ dbt.type_string() }} ) as diagnosis_code_25 + , cast(diagnosis_poa_1 as {{ dbt.type_string() }} ) as diagnosis_poa_1 + , cast(diagnosis_poa_2 as {{ dbt.type_string() }} ) as diagnosis_poa_2 + , cast(diagnosis_poa_3 as {{ dbt.type_string() }} ) as diagnosis_poa_3 + , cast(diagnosis_poa_4 as {{ dbt.type_string() }} ) as diagnosis_poa_4 + , cast(diagnosis_poa_5 as {{ dbt.type_string() }} ) as diagnosis_poa_5 + , cast(diagnosis_poa_6 as {{ dbt.type_string() }} ) as diagnosis_poa_6 + , cast(diagnosis_poa_7 as {{ dbt.type_string() }} ) as diagnosis_poa_7 + , cast(diagnosis_poa_8 as {{ dbt.type_string() }} ) as diagnosis_poa_8 + , cast(diagnosis_poa_9 as {{ dbt.type_string() }} ) as diagnosis_poa_9 + , cast(diagnosis_poa_10 as {{ dbt.type_string() }} ) as diagnosis_poa_10 + , cast(diagnosis_poa_11 as {{ dbt.type_string() }} ) as diagnosis_poa_11 + , cast(diagnosis_poa_12 as {{ dbt.type_string() }} ) as diagnosis_poa_12 + , cast(diagnosis_poa_13 as {{ dbt.type_string() }} ) as diagnosis_poa_13 + , cast(diagnosis_poa_14 as {{ dbt.type_string() }} ) as diagnosis_poa_14 + , cast(diagnosis_poa_15 as {{ dbt.type_string() }} ) as diagnosis_poa_15 + , cast(diagnosis_poa_16 as {{ dbt.type_string() }} ) as diagnosis_poa_16 + , cast(diagnosis_poa_17 as {{ dbt.type_string() }} ) as diagnosis_poa_17 + , cast(diagnosis_poa_18 as {{ dbt.type_string() }} ) as diagnosis_poa_18 + , cast(diagnosis_poa_19 as {{ dbt.type_string() }} ) as diagnosis_poa_19 + , cast(diagnosis_poa_20 as {{ dbt.type_string() }} ) as diagnosis_poa_20 + , cast(diagnosis_poa_21 as {{ dbt.type_string() }} ) as diagnosis_poa_21 + , cast(diagnosis_poa_22 as {{ dbt.type_string() }} ) as diagnosis_poa_22 + , cast(diagnosis_poa_23 as {{ dbt.type_string() }} ) as diagnosis_poa_23 + , cast(diagnosis_poa_24 as {{ dbt.type_string() }} ) as diagnosis_poa_24 + , cast(diagnosis_poa_25 as {{ dbt.type_string() }} ) as diagnosis_poa_25 + , cast(procedure_code_type as {{ dbt.type_string() }} ) as procedure_code_type + , cast(procedure_code_1 as {{ dbt.type_string() }} ) as procedure_code_1 + , cast(procedure_code_2 as {{ dbt.type_string() }} ) as procedure_code_2 + , cast(procedure_code_3 as {{ dbt.type_string() }} ) as procedure_code_3 + , cast(procedure_code_4 as {{ dbt.type_string() }} ) as procedure_code_4 + , cast(procedure_code_5 as {{ dbt.type_string() }} ) as procedure_code_5 + , cast(procedure_code_6 as {{ dbt.type_string() }} ) as procedure_code_6 + , cast(procedure_code_7 as {{ dbt.type_string() }} ) as procedure_code_7 + , cast(procedure_code_8 as {{ dbt.type_string() }} ) as procedure_code_8 + , cast(procedure_code_9 as {{ dbt.type_string() }} ) as procedure_code_9 + , cast(procedure_code_10 as {{ dbt.type_string() }} ) as procedure_code_10 + , cast(procedure_code_11 as {{ dbt.type_string() }} ) as procedure_code_11 + , cast(procedure_code_12 as {{ dbt.type_string() }} ) as procedure_code_12 + , cast(procedure_code_13 as {{ dbt.type_string() }} ) as procedure_code_13 + , cast(procedure_code_14 as {{ dbt.type_string() }} ) as procedure_code_14 + , cast(procedure_code_15 as {{ dbt.type_string() }} ) as procedure_code_15 + , cast(procedure_code_16 as {{ dbt.type_string() }} ) as procedure_code_16 + , cast(procedure_code_17 as {{ dbt.type_string() }} ) as procedure_code_17 + , cast(procedure_code_18 as {{ dbt.type_string() }} ) as procedure_code_18 + , cast(procedure_code_19 as {{ dbt.type_string() }} ) as procedure_code_19 + , cast(procedure_code_20 as {{ dbt.type_string() }} ) as procedure_code_20 + , cast(procedure_code_21 as {{ dbt.type_string() }} ) as procedure_code_21 + , cast(procedure_code_22 as {{ dbt.type_string() }} ) as procedure_code_22 + , cast(procedure_code_23 as {{ dbt.type_string() }} ) as procedure_code_23 + , cast(procedure_code_24 as {{ dbt.type_string() }} ) as procedure_code_24 + , cast(procedure_code_25 as {{ dbt.type_string() }} ) as procedure_code_25 + , {{ try_to_cast_date('procedure_date_1', 'YYYY-MM-DD') }} as procedure_date_1 + , {{ try_to_cast_date('procedure_date_2', 'YYYY-MM-DD') }} as procedure_date_2 + , {{ try_to_cast_date('procedure_date_3', 'YYYY-MM-DD') }} as procedure_date_3 + , {{ try_to_cast_date('procedure_date_4', 'YYYY-MM-DD') }} as procedure_date_4 + , {{ try_to_cast_date('procedure_date_5', 'YYYY-MM-DD') }} as procedure_date_5 + , {{ try_to_cast_date('procedure_date_6', 'YYYY-MM-DD') }} as procedure_date_6 + , {{ try_to_cast_date('procedure_date_7', 'YYYY-MM-DD') }} as procedure_date_7 + , {{ try_to_cast_date('procedure_date_8', 'YYYY-MM-DD') }} as procedure_date_8 + , {{ try_to_cast_date('procedure_date_9', 'YYYY-MM-DD') }} as procedure_date_9 + , {{ try_to_cast_date('procedure_date_10', 'YYYY-MM-DD') }} as procedure_date_10 + , {{ try_to_cast_date('procedure_date_11', 'YYYY-MM-DD') }} as procedure_date_11 + , {{ try_to_cast_date('procedure_date_12', 'YYYY-MM-DD') }} as procedure_date_12 + , {{ try_to_cast_date('procedure_date_13', 'YYYY-MM-DD') }} as procedure_date_13 + , {{ try_to_cast_date('procedure_date_14', 'YYYY-MM-DD') }} as procedure_date_14 + , {{ try_to_cast_date('procedure_date_15', 'YYYY-MM-DD') }} as procedure_date_15 + , {{ try_to_cast_date('procedure_date_16', 'YYYY-MM-DD') }} as procedure_date_16 + , {{ try_to_cast_date('procedure_date_17', 'YYYY-MM-DD') }} as procedure_date_17 + , {{ try_to_cast_date('procedure_date_18', 'YYYY-MM-DD') }} as procedure_date_18 + , {{ try_to_cast_date('procedure_date_19', 'YYYY-MM-DD') }} as procedure_date_19 + , {{ try_to_cast_date('procedure_date_20', 'YYYY-MM-DD') }} as procedure_date_20 + , {{ try_to_cast_date('procedure_date_21', 'YYYY-MM-DD') }} as procedure_date_21 + , {{ try_to_cast_date('procedure_date_22', 'YYYY-MM-DD') }} as procedure_date_22 + , {{ try_to_cast_date('procedure_date_23', 'YYYY-MM-DD') }} as procedure_date_23 + , {{ try_to_cast_date('procedure_date_24', 'YYYY-MM-DD') }} as procedure_date_24 + , {{ try_to_cast_date('procedure_date_25', 'YYYY-MM-DD') }} as procedure_date_25 + , cast(in_network_flag as integer) as in_network_flag + , cast(data_source as {{ dbt.type_string() }} ) as data_source + , cast(file_name as {{ dbt.type_string() }} ) as file_name + , cast(ingest_datetime as {{ dbt.type_string() }} ) as ingest_datetime + from mapping + +) + +select + claim_id + , claim_line_number + , claim_type + , patient_id + , member_id + , payer + , {{ the_tuva_project.quote_column('plan') }} + , claim_start_date + , claim_end_date + , claim_line_start_date + , claim_line_end_date + , admission_date + , discharge_date + , admit_source_code + , admit_type_code + , discharge_disposition_code + , place_of_service_code + , bill_type_code + , ms_drg_code + , apr_drg_code + , revenue_center_code + , service_unit_quantity + , hcpcs_code + , hcpcs_modifier_1 + , hcpcs_modifier_2 + , hcpcs_modifier_3 + , hcpcs_modifier_4 + , hcpcs_modifier_5 + , rendering_npi + , rendering_tin + , billing_npi + , billing_tin + , facility_npi + , paid_date + , paid_amount + , allowed_amount + , charge_amount + , coinsurance_amount + , copayment_amount + , deductible_amount + , total_cost_amount + , diagnosis_code_type + , diagnosis_code_1 + , diagnosis_code_2 + , diagnosis_code_3 + , diagnosis_code_4 + , diagnosis_code_5 + , diagnosis_code_6 + , diagnosis_code_7 + , diagnosis_code_8 + , diagnosis_code_9 + , diagnosis_code_10 + , diagnosis_code_11 + , diagnosis_code_12 + , diagnosis_code_13 + , diagnosis_code_14 + , diagnosis_code_15 + , diagnosis_code_16 + , diagnosis_code_17 + , diagnosis_code_18 + , diagnosis_code_19 + , diagnosis_code_20 + , diagnosis_code_21 + , diagnosis_code_22 + , diagnosis_code_23 + , diagnosis_code_24 + , diagnosis_code_25 + , diagnosis_poa_1 + , diagnosis_poa_2 + , diagnosis_poa_3 + , diagnosis_poa_4 + , diagnosis_poa_5 + , diagnosis_poa_6 + , diagnosis_poa_7 + , diagnosis_poa_8 + , diagnosis_poa_9 + , diagnosis_poa_10 + , diagnosis_poa_11 + , diagnosis_poa_12 + , diagnosis_poa_13 + , diagnosis_poa_14 + , diagnosis_poa_15 + , diagnosis_poa_16 + , diagnosis_poa_17 + , diagnosis_poa_18 + , diagnosis_poa_19 + , diagnosis_poa_20 + , diagnosis_poa_21 + , diagnosis_poa_22 + , diagnosis_poa_23 + , diagnosis_poa_24 + , diagnosis_poa_25 + , procedure_code_type + , procedure_code_1 + , procedure_code_2 + , procedure_code_3 + , procedure_code_4 + , procedure_code_5 + , procedure_code_6 + , procedure_code_7 + , procedure_code_8 + , procedure_code_9 + , procedure_code_10 + , procedure_code_11 + , procedure_code_12 + , procedure_code_13 + , procedure_code_14 + , procedure_code_15 + , procedure_code_16 + , procedure_code_17 + , procedure_code_18 + , procedure_code_19 + , procedure_code_20 + , procedure_code_21 + , procedure_code_22 + , procedure_code_23 + , procedure_code_24 + , procedure_code_25 + , procedure_date_1 + , procedure_date_2 + , procedure_date_3 + , procedure_date_4 + , procedure_date_5 + , procedure_date_6 + , procedure_date_7 + , procedure_date_8 + , procedure_date_9 + , procedure_date_10 + , procedure_date_11 + , procedure_date_12 + , procedure_date_13 + , procedure_date_14 + , procedure_date_15 + , procedure_date_16 + , procedure_date_17 + , procedure_date_18 + , procedure_date_19 + , procedure_date_20 + , procedure_date_21 + , procedure_date_22 + , procedure_date_23 + , procedure_date_24 + , procedure_date_25 + , in_network_flag + , data_source + , file_name + , ingest_datetime +from add_data_types \ No newline at end of file diff --git a/models/intermediate/int_pharmacy_claim_adr.sql b/models/intermediate/int_pharmacy_claim_adr.sql new file mode 100644 index 00000000..7db2b004 --- /dev/null +++ b/models/intermediate/int_pharmacy_claim_adr.sql @@ -0,0 +1,197 @@ +with staged_data as ( + + select + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_line_ndc_cd + , clm_type_cd + , clm_line_from_dt + , prvdr_srvc_id_qlfyr_cd + , clm_srvc_prvdr_gnrc_id_num + , clm_dspnsng_stus_cd + , clm_daw_prod_slctn_cd + , clm_line_srvc_unit_qty + , clm_line_days_suply_qty + , prvdr_prsbng_id_qlfyr_cd + , clm_prsbng_prvdr_gnrc_id_num + , clm_line_bene_pmt_amt + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_idr_ld_dt + , clm_line_rx_srvc_rfrnc_num + , clm_line_rx_fill_num + , clm_phrmcy_srvc_type_cd + , file_name + , file_date + from {{ ref('stg_partd_claims') }} + +) + +, beneficiary_xref as ( + + select * from {{ ref('int_beneficiary_xref_deduped') }} + +) + +/* + dedupe full rows that may appear in multiple files + source fields not mapped or used for adjustment logic are commented out +*/ +, add_row_num as ( + + select *, row_number() over ( + partition by + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_line_ndc_cd + /*, clm_type_cd*/ + , clm_line_from_dt + , prvdr_srvc_id_qlfyr_cd + , clm_srvc_prvdr_gnrc_id_num + , clm_dspnsng_stus_cd + /*, clm_daw_prod_slctn_cd*/ + , clm_line_srvc_unit_qty + , clm_line_days_suply_qty + , prvdr_prsbng_id_qlfyr_cd + , clm_prsbng_prvdr_gnrc_id_num + , clm_line_bene_pmt_amt + , clm_adjsmt_type_cd + /*, clm_efctv_dt*/ + /*, clm_idr_ld_dt*/ + , clm_line_rx_srvc_rfrnc_num + , clm_line_rx_fill_num + /*, clm_phrmcy_srvc_type_cd*/ + order by file_date desc + ) as row_num + from staged_data + +) + +, dedupe as ( + + select + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_line_ndc_cd + , clm_line_from_dt + , prvdr_srvc_id_qlfyr_cd + , clm_srvc_prvdr_gnrc_id_num + , clm_dspnsng_stus_cd + , clm_line_srvc_unit_qty + , clm_line_days_suply_qty + , prvdr_prsbng_id_qlfyr_cd + , clm_prsbng_prvdr_gnrc_id_num + , clm_line_bene_pmt_amt + , clm_adjsmt_type_cd + , clm_line_rx_srvc_rfrnc_num + , clm_line_rx_fill_num + , file_name + , file_date + from add_row_num + where row_num = 1 + +) + +/* coalesce current MBI from XREF if exists and MBI on claim */ +, add_current_mbi as ( + + select + dedupe.cur_clm_uniq_id + , coalesce(beneficiary_xref.crnt_num, dedupe.bene_mbi_id) as current_bene_mbi_id + , dedupe.bene_hic_num + , dedupe.clm_line_ndc_cd + , dedupe.clm_line_from_dt + , dedupe.prvdr_srvc_id_qlfyr_cd + , dedupe.clm_srvc_prvdr_gnrc_id_num + , dedupe.clm_dspnsng_stus_cd + , dedupe.clm_line_srvc_unit_qty + , dedupe.clm_line_days_suply_qty + , dedupe.prvdr_prsbng_id_qlfyr_cd + , dedupe.clm_prsbng_prvdr_gnrc_id_num + , dedupe.clm_line_bene_pmt_amt + , dedupe.clm_adjsmt_type_cd + , dedupe.clm_line_rx_srvc_rfrnc_num + , dedupe.clm_line_rx_fill_num + , dedupe.file_name + , dedupe.file_date + from dedupe + left join beneficiary_xref + on dedupe.bene_mbi_id = beneficiary_xref.prvs_num + +) + +/* + apply adjustment logic by grouping part D claims by their natural keys: + - CLM_LINE_FROM_DT + - PRVDR_SRVC_ID_QLFYR_CD + - CLM_SRVC_PRVDR_GNRC_ID_NUM + - CLM_DSPNSNG_STUS_CD + - CLM_LINE_RX_SRVC_RFRNC_NUM + - CLM_LINE_RX_FILL_NUM + + then sorting them by the CLM_ADJSMT_TYPE_CD code + 0 = Original Claim + 1 = Cancellation Claim + 2 = Adjustment claim + + final filtering takes place in dedupe model +*/ +, sort_adjusted_claims as ( + + select + cur_clm_uniq_id + , current_bene_mbi_id + , bene_hic_num + , clm_line_ndc_cd + , clm_line_from_dt + , prvdr_srvc_id_qlfyr_cd + , clm_srvc_prvdr_gnrc_id_num + , clm_dspnsng_stus_cd + , clm_line_srvc_unit_qty + , clm_line_days_suply_qty + , prvdr_prsbng_id_qlfyr_cd + , clm_prsbng_prvdr_gnrc_id_num + , clm_line_bene_pmt_amt + , clm_adjsmt_type_cd + , clm_line_rx_srvc_rfrnc_num + , clm_line_rx_fill_num + , file_name + , file_date + , row_number() over ( + partition by + clm_line_from_dt + , prvdr_srvc_id_qlfyr_cd + , clm_srvc_prvdr_gnrc_id_num + , clm_dspnsng_stus_cd + , clm_line_rx_srvc_rfrnc_num + , clm_line_rx_fill_num + order by clm_adjsmt_type_cd desc + ) as row_num + from add_current_mbi + +) + +select + cur_clm_uniq_id + , current_bene_mbi_id + , bene_hic_num + , clm_line_ndc_cd + , clm_line_from_dt + , prvdr_srvc_id_qlfyr_cd + , clm_srvc_prvdr_gnrc_id_num + , clm_dspnsng_stus_cd + , clm_line_srvc_unit_qty + , clm_line_days_suply_qty + , prvdr_prsbng_id_qlfyr_cd + , clm_prsbng_prvdr_gnrc_id_num + , clm_line_bene_pmt_amt + , clm_adjsmt_type_cd + , clm_line_rx_srvc_rfrnc_num + , clm_line_rx_fill_num + , file_name + , file_date + , row_num +from sort_adjusted_claims \ No newline at end of file diff --git a/models/intermediate/int_pharmacy_claim_deduped.sql b/models/intermediate/int_pharmacy_claim_deduped.sql new file mode 100644 index 00000000..0f2eec8b --- /dev/null +++ b/models/intermediate/int_pharmacy_claim_deduped.sql @@ -0,0 +1,151 @@ +with sort_adjusted_claims as ( + + select + cur_clm_uniq_id + , current_bene_mbi_id + , bene_hic_num + , clm_line_ndc_cd + , clm_line_from_dt + , prvdr_srvc_id_qlfyr_cd + , clm_srvc_prvdr_gnrc_id_num + , clm_dspnsng_stus_cd + , clm_line_srvc_unit_qty + , clm_line_days_suply_qty + , prvdr_prsbng_id_qlfyr_cd + , clm_prsbng_prvdr_gnrc_id_num + , clm_line_bene_pmt_amt + , clm_adjsmt_type_cd + , clm_line_rx_srvc_rfrnc_num + , clm_line_rx_fill_num + , file_name + , file_date + , row_num + from {{ ref('int_pharmacy_claim_adr') }} + +) + +/* + apply final adjustment logic by selecting latest version of claim + and removing cancelled claims + + CLM_ADJSMT_TYPE_CD: + 0 = Original Claim + 1 = Cancellation Claim + 2 = Adjustment claim +*/ +, filter_claims as ( + + select + cur_clm_uniq_id + , current_bene_mbi_id + , bene_hic_num + , clm_line_ndc_cd + , clm_line_from_dt + , prvdr_srvc_id_qlfyr_cd + , clm_srvc_prvdr_gnrc_id_num + , clm_dspnsng_stus_cd + , clm_line_srvc_unit_qty + , clm_line_days_suply_qty + , prvdr_prsbng_id_qlfyr_cd + , clm_prsbng_prvdr_gnrc_id_num + , clm_line_bene_pmt_amt + , clm_adjsmt_type_cd + , clm_line_rx_srvc_rfrnc_num + , clm_line_rx_fill_num + , file_name + , file_date + , row_num + from sort_adjusted_claims + where row_num = 1 + and clm_adjsmt_type_cd <> '1' + +) + +/* + remove claim lines where claim ID+line number not unique + even after adjustments have been applied +*/ +, claim_dupes as ( + + select cur_clm_uniq_id + from filter_claims + group by cur_clm_uniq_id + having count(*) > 1 + +) + +, remove_dupes as ( + + select filter_claims.* + from filter_claims + left join claim_dupes + on filter_claims.cur_clm_uniq_id = claim_dupes.cur_clm_uniq_id + where claim_dupes.cur_clm_uniq_id is null + +) + +, mapping as ( + + select + cur_clm_uniq_id as claim_id + , 1 as claim_line_number + , current_bene_mbi_id as patient_id + , current_bene_mbi_id as member_id + , cast('medicare' as {{ dbt.type_string() }} ) as payer + , cast('medicare'as {{ dbt.type_string() }} ) as {{ the_tuva_project.quote_column('plan') }} + , case + when prvdr_prsbng_id_qlfyr_cd in ('1', '01') + then clm_prsbng_prvdr_gnrc_id_num + else null + end as prescribing_provider_npi + , case + when prvdr_srvc_id_qlfyr_cd in ('1', '01') + then clm_srvc_prvdr_gnrc_id_num + else null + end as dispensing_provider_npi + , clm_line_from_dt as dispensing_date + , clm_line_ndc_cd as ndc_code + , clm_line_srvc_unit_qty as quantity + , clm_line_days_suply_qty as days_supply + , clm_line_rx_fill_num as refills + , clm_line_from_dt as paid_date + , clm_line_bene_pmt_amt as paid_amount + , cast(null as {{ dbt.type_string() }} ) as allowed_amount + , cast(null as {{ dbt.type_string() }} ) as charge_amount + , cast(null as {{ dbt.type_string() }} ) as coinsurance_amount + , clm_line_bene_pmt_amt as copayment_amount + , cast(null as {{ dbt.type_string() }} ) as deductible_amount + , 1 as in_network_flag + , cast('medicare cclf' as {{ dbt.type_string() }} ) as data_source + , file_name as file_name + , file_date as ingest_datetime + from remove_dupes + +) + +select + claim_id + , claim_line_number + , patient_id + , member_id + , payer + , {{ the_tuva_project.quote_column('plan') }} + , prescribing_provider_npi + , dispensing_provider_npi + , dispensing_date + , ndc_code + , quantity + , days_supply + , refills + , paid_date + , paid_amount + , allowed_amount + , charge_amount + , coinsurance_amount + , copayment_amount + , deductible_amount + , in_network_flag + , data_source + , file_name + , ingest_datetime +from mapping \ No newline at end of file diff --git a/models/intermediate/int_physician_claim_adr.sql b/models/intermediate/int_physician_claim_adr.sql new file mode 100644 index 00000000..e2fecc4d --- /dev/null +++ b/models/intermediate/int_physician_claim_adr.sql @@ -0,0 +1,344 @@ +with staged_data as ( + + select + cur_clm_uniq_id + , clm_line_num + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_from_dt + , clm_thru_dt + , rndrg_prvdr_type_cd + , rndrg_prvdr_fips_st_cd + , clm_prvdr_spclty_cd + , clm_fed_type_srvc_cd + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + , clm_line_prmry_pyr_cd + , clm_line_dgns_cd + , clm_rndrg_prvdr_tax_num + , rndrg_prvdr_npi_num + , clm_carr_pmt_dnl_cd + , clm_prcsg_ind_cd + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_idr_ld_dt + , clm_cntl_num + , bene_eqtbl_bic_hicn_num + , clm_line_alowd_chrg_amt + , clm_line_srvc_unit_qty + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_disp_cd + , clm_dgns_1_cd + , clm_dgns_2_cd + , clm_dgns_3_cd + , clm_dgns_4_cd + , clm_dgns_5_cd + , clm_dgns_6_cd + , clm_dgns_7_cd + , clm_dgns_8_cd + , dgns_prcdr_icd_ind + , clm_dgns_9_cd + , clm_dgns_10_cd + , clm_dgns_11_cd + , clm_dgns_12_cd + , hcpcs_betos_cd + , file_name + , file_date + from {{ ref('stg_partb_physicians') }} + +) + +, beneficiary_xref as ( + + select * from {{ ref('int_beneficiary_xref_deduped') }} + +) + +/* dedupe full rows that may appear in multiple files */ +, add_row_num as ( + + select *, row_number() over ( + partition by + cur_clm_uniq_id + , clm_line_num + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_from_dt + , clm_thru_dt + , rndrg_prvdr_type_cd + , rndrg_prvdr_fips_st_cd + , clm_prvdr_spclty_cd + , clm_fed_type_srvc_cd + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + , clm_line_prmry_pyr_cd + , clm_line_dgns_cd + , clm_rndrg_prvdr_tax_num + , rndrg_prvdr_npi_num + , clm_carr_pmt_dnl_cd + , clm_prcsg_ind_cd + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_idr_ld_dt + , clm_cntl_num + , bene_eqtbl_bic_hicn_num + , clm_line_alowd_chrg_amt + , clm_line_srvc_unit_qty + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_disp_cd + , clm_dgns_1_cd + , clm_dgns_2_cd + , clm_dgns_3_cd + , clm_dgns_4_cd + , clm_dgns_5_cd + , clm_dgns_6_cd + , clm_dgns_7_cd + , clm_dgns_8_cd + , dgns_prcdr_icd_ind + , clm_dgns_9_cd + , clm_dgns_10_cd + , clm_dgns_11_cd + , clm_dgns_12_cd + , hcpcs_betos_cd + order by file_date desc + ) as row_num + from staged_data + +) + +/* source fields not mapped or used for adjustment logic are commented out */ +, dedupe as ( + + select + cur_clm_uniq_id + , clm_line_num + , bene_mbi_id + /*, bene_hic_num*/ + /*, clm_type_cd*/ + , clm_from_dt + , clm_thru_dt + /*, rndrg_prvdr_type_cd*/ + /*, rndrg_prvdr_fips_st_cd*/ + /*, clm_prvdr_spclty_cd*/ + /*, clm_fed_type_srvc_cd*/ + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + /*, clm_line_prmry_pyr_cd*/ + /*, clm_line_dgns_cd*/ + , clm_rndrg_prvdr_tax_num + , rndrg_prvdr_npi_num + /*, clm_carr_pmt_dnl_cd*/ + /*, clm_prcsg_ind_cd*/ + , clm_adjsmt_type_cd + , clm_efctv_dt + /*, clm_idr_ld_dt*/ + , clm_cntl_num + /*, bene_eqtbl_bic_hicn_num*/ + , clm_line_alowd_chrg_amt + , clm_line_srvc_unit_qty + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + /*, clm_disp_cd*/ + , clm_dgns_1_cd + , clm_dgns_2_cd + , clm_dgns_3_cd + , clm_dgns_4_cd + , clm_dgns_5_cd + , clm_dgns_6_cd + , clm_dgns_7_cd + , clm_dgns_8_cd + , dgns_prcdr_icd_ind + , clm_dgns_9_cd + , clm_dgns_10_cd + , clm_dgns_11_cd + , clm_dgns_12_cd + /*, hcpcs_betos_cd*/ + , file_name + , file_date + from add_row_num + where row_num = 1 + +) + +/* coalesce current MBI from XREF if exists and MBI on claim */ +, add_current_mbi as ( + + select + dedupe.cur_clm_uniq_id + , dedupe.clm_line_num + , coalesce(beneficiary_xref.crnt_num, dedupe.bene_mbi_id) as current_bene_mbi_id + , dedupe.clm_from_dt + , dedupe.clm_thru_dt + , dedupe.clm_pos_cd + , dedupe.clm_line_from_dt + , dedupe.clm_line_thru_dt + , dedupe.clm_line_hcpcs_cd + , dedupe.clm_line_cvrd_pd_amt + , dedupe.clm_rndrg_prvdr_tax_num + , dedupe.rndrg_prvdr_npi_num + , dedupe.clm_adjsmt_type_cd + , dedupe.clm_efctv_dt + , dedupe.clm_cntl_num + , dedupe.clm_line_alowd_chrg_amt + , dedupe.clm_line_srvc_unit_qty + , dedupe.hcpcs_1_mdfr_cd + , dedupe.hcpcs_2_mdfr_cd + , dedupe.hcpcs_3_mdfr_cd + , dedupe.hcpcs_4_mdfr_cd + , dedupe.hcpcs_5_mdfr_cd + , dedupe.clm_dgns_1_cd + , dedupe.clm_dgns_2_cd + , dedupe.clm_dgns_3_cd + , dedupe.clm_dgns_4_cd + , dedupe.clm_dgns_5_cd + , dedupe.clm_dgns_6_cd + , dedupe.clm_dgns_7_cd + , dedupe.clm_dgns_8_cd + , dedupe.dgns_prcdr_icd_ind + , dedupe.clm_dgns_9_cd + , dedupe.clm_dgns_10_cd + , dedupe.clm_dgns_11_cd + , dedupe.clm_dgns_12_cd + , dedupe.file_name + , dedupe.file_date + from dedupe + left join beneficiary_xref + on dedupe.bene_mbi_id = beneficiary_xref.prvs_num + + +) + +/* + 1) apply adjustment logic by grouping part B Physician claims by their natural keys: + - CLM_CNTL_NUM + - Most Recent MBI + - CLM_LINE_NUM (not listed in CCLF docs, but used to prevent line detail loss) + + 2) sort grouped claims by the latest CLM_EFCTV_DT and CUR_CLM_UNIQ_ID since CLM_ADJSMT_TYPE_CD + is not used consistently to indicate the latest final version of an adjusted claim. + + 3) change paid amounts to negative for canceled claims + + (CCLF docs ref: 5.3 Calculating Beneficiary-Level Expenditures) +*/ +, sort_adjusted_claims as ( + + select + cur_clm_uniq_id + , clm_line_num + , current_bene_mbi_id + , clm_from_dt + , clm_thru_dt + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , case + when clm_adjsmt_type_cd = '1' then {{ cast_numeric('clm_line_cvrd_pd_amt') }} * -1 + else {{ cast_numeric('clm_line_cvrd_pd_amt') }} + end as clm_line_cvrd_pd_amt + , clm_rndrg_prvdr_tax_num + , rndrg_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_cntl_num + , case + when clm_adjsmt_type_cd = '1' then {{ cast_numeric('clm_line_alowd_chrg_amt') }} * -1 + else {{ cast_numeric('clm_line_alowd_chrg_amt') }} + end as clm_line_alowd_chrg_amt + , clm_line_srvc_unit_qty + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_dgns_1_cd + , clm_dgns_2_cd + , clm_dgns_3_cd + , clm_dgns_4_cd + , clm_dgns_5_cd + , clm_dgns_6_cd + , clm_dgns_7_cd + , clm_dgns_8_cd + , dgns_prcdr_icd_ind + , clm_dgns_9_cd + , clm_dgns_10_cd + , clm_dgns_11_cd + , clm_dgns_12_cd + , file_name + , file_date + , row_number() over ( + partition by + clm_cntl_num + , clm_line_num + , current_bene_mbi_id + order by + clm_efctv_dt desc + , cur_clm_uniq_id desc + ) as row_num + from add_current_mbi + +) + +select + cur_clm_uniq_id + , clm_line_num + , current_bene_mbi_id + , clm_from_dt + , clm_thru_dt + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + , clm_rndrg_prvdr_tax_num + , rndrg_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_cntl_num + , clm_line_alowd_chrg_amt + , clm_line_srvc_unit_qty + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_dgns_1_cd + , clm_dgns_2_cd + , clm_dgns_3_cd + , clm_dgns_4_cd + , clm_dgns_5_cd + , clm_dgns_6_cd + , clm_dgns_7_cd + , clm_dgns_8_cd + , dgns_prcdr_icd_ind + , clm_dgns_9_cd + , clm_dgns_10_cd + , clm_dgns_11_cd + , clm_dgns_12_cd + , file_name + , file_date + , row_num +from sort_adjusted_claims \ No newline at end of file diff --git a/models/intermediate/int_physician_claim_deduped.sql b/models/intermediate/int_physician_claim_deduped.sql new file mode 100644 index 00000000..2195d058 --- /dev/null +++ b/models/intermediate/int_physician_claim_deduped.sql @@ -0,0 +1,620 @@ +with sort_adjusted_claims as ( + + select + cur_clm_uniq_id + , clm_line_num + , current_bene_mbi_id + , clm_from_dt + , clm_thru_dt + , clm_pos_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_hcpcs_cd + , clm_line_cvrd_pd_amt + , clm_rndrg_prvdr_tax_num + , rndrg_prvdr_npi_num + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_cntl_num + , clm_line_alowd_chrg_amt + , clm_line_srvc_unit_qty + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_dgns_1_cd + , clm_dgns_2_cd + , clm_dgns_3_cd + , clm_dgns_4_cd + , clm_dgns_5_cd + , clm_dgns_6_cd + , clm_dgns_7_cd + , clm_dgns_8_cd + , dgns_prcdr_icd_ind + , clm_dgns_9_cd + , clm_dgns_10_cd + , clm_dgns_11_cd + , clm_dgns_12_cd + , file_name + , file_date + , row_num + from {{ ref('int_physician_claim_adr') }} + +) + +/* + sum the adjusted line amounts + + (CCLF docs ref: 5.3 Calculating Beneficiary-Level Expenditures) +*/ +, line_totals as ( + + select + clm_cntl_num + , clm_line_num + , current_bene_mbi_id + , sum(clm_line_cvrd_pd_amt) as sum_clm_line_cvrd_pd_amt + , sum(clm_line_alowd_chrg_amt) as sum_clm_line_alowd_chrg_amt + from sort_adjusted_claims + group by + clm_cntl_num + , clm_line_num + , current_bene_mbi_id + +) + +/* + apply final adjustment logic by selecting latest version of claim, + removing any remaining claims with a canceled status, and adding line totals. +*/ +, filter_claims as ( + + select + sort_adjusted_claims.cur_clm_uniq_id + , sort_adjusted_claims.clm_line_num + , sort_adjusted_claims.current_bene_mbi_id + , sort_adjusted_claims.clm_from_dt + , sort_adjusted_claims.clm_thru_dt + , sort_adjusted_claims.clm_pos_cd + , sort_adjusted_claims.clm_line_from_dt + , sort_adjusted_claims.clm_line_thru_dt + , sort_adjusted_claims.clm_line_hcpcs_cd + , line_totals.sum_clm_line_cvrd_pd_amt as clm_line_cvrd_pd_amt + , sort_adjusted_claims.clm_rndrg_prvdr_tax_num + , sort_adjusted_claims.rndrg_prvdr_npi_num + , sort_adjusted_claims.clm_adjsmt_type_cd + , sort_adjusted_claims.clm_efctv_dt + , sort_adjusted_claims.clm_cntl_num + , line_totals.sum_clm_line_alowd_chrg_amt as clm_line_alowd_chrg_amt + , sort_adjusted_claims.clm_line_srvc_unit_qty + , sort_adjusted_claims.hcpcs_1_mdfr_cd + , sort_adjusted_claims.hcpcs_2_mdfr_cd + , sort_adjusted_claims.hcpcs_3_mdfr_cd + , sort_adjusted_claims.hcpcs_4_mdfr_cd + , sort_adjusted_claims.hcpcs_5_mdfr_cd + , sort_adjusted_claims.clm_dgns_1_cd + , sort_adjusted_claims.clm_dgns_2_cd + , sort_adjusted_claims.clm_dgns_3_cd + , sort_adjusted_claims.clm_dgns_4_cd + , sort_adjusted_claims.clm_dgns_5_cd + , sort_adjusted_claims.clm_dgns_6_cd + , sort_adjusted_claims.clm_dgns_7_cd + , sort_adjusted_claims.clm_dgns_8_cd + , sort_adjusted_claims.dgns_prcdr_icd_ind + , sort_adjusted_claims.clm_dgns_9_cd + , sort_adjusted_claims.clm_dgns_10_cd + , sort_adjusted_claims.clm_dgns_11_cd + , sort_adjusted_claims.clm_dgns_12_cd + , sort_adjusted_claims.file_name + , sort_adjusted_claims.file_date + from sort_adjusted_claims + left join line_totals + on sort_adjusted_claims.clm_cntl_num = line_totals.clm_cntl_num + and sort_adjusted_claims.clm_line_num = line_totals.clm_line_num + and sort_adjusted_claims.current_bene_mbi_id = line_totals.current_bene_mbi_id + where sort_adjusted_claims.row_num = 1 + and sort_adjusted_claims.clm_adjsmt_type_cd <> '1' + +) + +/* + remove claim lines where claim ID+line number not unique + even after adjustments have been applied +*/ +, claim_dupes as ( + + select cur_clm_uniq_id, clm_line_num + from filter_claims + group by cur_clm_uniq_id, clm_line_num + having count(*) > 1 + +) + +, remove_dupes as ( + + select filter_claims.* + from filter_claims + left join claim_dupes + on filter_claims.cur_clm_uniq_id = claim_dupes.cur_clm_uniq_id + and filter_claims.clm_line_num = claim_dupes.clm_line_num + where claim_dupes.cur_clm_uniq_id is null + +) + +, mapping as ( + + select + cur_clm_uniq_id as claim_id + , clm_line_num as claim_line_number + , cast('professional' as {{ dbt.type_string() }} ) as claim_type + , current_bene_mbi_id as patient_id + , current_bene_mbi_id as member_id + , cast('medicare' as {{ dbt.type_string() }} ) as payer + , cast('medicare' as {{ dbt.type_string() }} ) as {{ the_tuva_project.quote_column('plan') }} + , case + when clm_from_dt in ('1000-01-01', '9999-12-31') then null + else clm_from_dt + end as claim_start_date + , case + when clm_thru_dt in ('1000-01-01', '9999-12-31') then null + else clm_thru_dt + end as claim_end_date + , case + when clm_line_from_dt in ('1000-01-01', '9999-12-31') then null + else clm_line_from_dt + end as claim_line_start_date + , case + when clm_line_thru_dt in ('1000-01-01', '9999-12-31') then null + else clm_line_thru_dt + end as claim_line_end_date + , cast(null as date) as admission_date + , cast(null as date) as discharge_date + , cast(null as {{ dbt.type_string() }} ) as admit_source_code + , cast(null as {{ dbt.type_string() }} ) as admit_type_code + , cast(null as {{ dbt.type_string() }} ) as discharge_disposition_code + , clm_pos_cd as place_of_service_code + , cast(null as {{ dbt.type_string() }} ) as bill_type_code + , cast(null as {{ dbt.type_string() }} ) as ms_drg_code + , cast(null as {{ dbt.type_string() }} ) as apr_drg_code + , cast(null as {{ dbt.type_string() }} ) as revenue_center_code + , clm_line_srvc_unit_qty as service_unit_quantity + , clm_line_hcpcs_cd as hcpcs_code + , hcpcs_1_mdfr_cd as hcpcs_modifier_1 + , hcpcs_2_mdfr_cd as hcpcs_modifier_2 + , hcpcs_3_mdfr_cd as hcpcs_modifier_3 + , hcpcs_4_mdfr_cd as hcpcs_modifier_4 + , hcpcs_5_mdfr_cd as hcpcs_modifier_5 + , rndrg_prvdr_npi_num as rendering_npi + , clm_rndrg_prvdr_tax_num as rendering_tin + , cast(null as {{ dbt.type_string() }} ) as billing_npi + , cast(null as {{ dbt.type_string() }} ) as billing_tin + , cast(null as {{ dbt.type_string() }} ) as facility_npi + , case + when clm_efctv_dt in ('1000-01-01', '9999-12-31') then null + else clm_efctv_dt + end as paid_date + , clm_line_cvrd_pd_amt as paid_amount + , clm_line_alowd_chrg_amt as allowed_amount + , clm_line_alowd_chrg_amt as charge_amount + , cast(null as {{ dbt.type_string() }} ) as coinsurance_amount + , cast(null as {{ dbt.type_string() }} ) as copayment_amount + , cast(null as {{ dbt.type_string() }} ) as deductible_amount + , cast(null as {{ dbt.type_string() }} ) as total_cost_amount + , case + when cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) = '0' then 'icd-10-cm' + when cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) = '9' then 'icd-9-cm' + else cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) + end as diagnosis_code_type + , clm_dgns_1_cd as diagnosis_code_1 + , clm_dgns_2_cd as diagnosis_code_2 + , clm_dgns_3_cd as diagnosis_code_3 + , clm_dgns_4_cd as diagnosis_code_4 + , clm_dgns_5_cd as diagnosis_code_5 + , clm_dgns_6_cd as diagnosis_code_6 + , clm_dgns_7_cd as diagnosis_code_7 + , clm_dgns_8_cd as diagnosis_code_8 + , clm_dgns_9_cd as diagnosis_code_9 + , clm_dgns_10_cd as diagnosis_code_10 + , clm_dgns_11_cd as diagnosis_code_11 + , clm_dgns_12_cd as diagnosis_code_12 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_13 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_14 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_15 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_16 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_17 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_18 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_19 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_20 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_21 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_22 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_23 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_24 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_code_25 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_1 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_2 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_3 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_4 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_5 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_6 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_7 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_8 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_9 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_10 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_11 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_12 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_13 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_14 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_15 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_16 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_17 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_18 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_19 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_20 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_21 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_22 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_23 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_24 + , cast(null as {{ dbt.type_string() }} ) as diagnosis_poa_25 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_type + , cast(null as {{ dbt.type_string() }} ) as procedure_code_1 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_2 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_3 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_4 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_5 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_6 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_7 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_8 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_9 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_10 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_11 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_12 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_13 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_14 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_15 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_16 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_17 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_18 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_19 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_20 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_21 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_22 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_23 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_24 + , cast(null as {{ dbt.type_string() }} ) as procedure_code_25 + , cast(null as date) as procedure_date_1 + , cast(null as date) as procedure_date_2 + , cast(null as date) as procedure_date_3 + , cast(null as date) as procedure_date_4 + , cast(null as date) as procedure_date_5 + , cast(null as date) as procedure_date_6 + , cast(null as date) as procedure_date_7 + , cast(null as date) as procedure_date_8 + , cast(null as date) as procedure_date_9 + , cast(null as date) as procedure_date_10 + , cast(null as date) as procedure_date_11 + , cast(null as date) as procedure_date_12 + , cast(null as date) as procedure_date_13 + , cast(null as date) as procedure_date_14 + , cast(null as date) as procedure_date_15 + , cast(null as date) as procedure_date_16 + , cast(null as date) as procedure_date_17 + , cast(null as date) as procedure_date_18 + , cast(null as date) as procedure_date_19 + , cast(null as date) as procedure_date_20 + , cast(null as date) as procedure_date_21 + , cast(null as date) as procedure_date_22 + , cast(null as date) as procedure_date_23 + , cast(null as date) as procedure_date_24 + , cast(null as date) as procedure_date_25 + , 1 as in_network_flag + , cast('medicare cclf' as {{ dbt.type_string() }} ) as data_source + , file_name + , file_date as ingest_datetime + from remove_dupes + +) + +, add_data_types as ( + + select + cast(claim_id as {{ dbt.type_string() }} ) as claim_id + , cast(claim_line_number as integer) as claim_line_number + , cast(claim_type as {{ dbt.type_string() }} ) as claim_type + , cast(patient_id as {{ dbt.type_string() }} ) as patient_id + , cast(member_id as {{ dbt.type_string() }} ) as member_id + , cast(payer as {{ dbt.type_string() }} ) as payer + , cast({{ the_tuva_project.quote_column('plan') }} as {{ dbt.type_string() }} ) as {{ the_tuva_project.quote_column('plan') }} + , {{ try_to_cast_date('claim_start_date', 'YYYY-MM-DD') }} as claim_start_date + , {{ try_to_cast_date('claim_end_date', 'YYYY-MM-DD') }} as claim_end_date + , {{ try_to_cast_date('claim_line_start_date', 'YYYY-MM-DD') }} as claim_line_start_date + , {{ try_to_cast_date('claim_line_end_date', 'YYYY-MM-DD') }} as claim_line_end_date + , {{ try_to_cast_date('admission_date', 'YYYY-MM-DD') }} as admission_date + , {{ try_to_cast_date('discharge_date', 'YYYY-MM-DD') }} as discharge_date + , cast(admit_source_code as {{ dbt.type_string() }} ) as admit_source_code + , cast(admit_type_code as {{ dbt.type_string() }} ) as admit_type_code + , cast(discharge_disposition_code as {{ dbt.type_string() }} ) as discharge_disposition_code + , cast(place_of_service_code as {{ dbt.type_string() }} ) as place_of_service_code + , cast(bill_type_code as {{ dbt.type_string() }} ) as bill_type_code + , cast(ms_drg_code as {{ dbt.type_string() }} ) as ms_drg_code + , cast(apr_drg_code as {{ dbt.type_string() }} ) as apr_drg_code + , cast(revenue_center_code as {{ dbt.type_string() }} ) as revenue_center_code + , {{ cast_numeric('service_unit_quantity') }} as service_unit_quantity + , cast(hcpcs_code as {{ dbt.type_string() }} ) as hcpcs_code + , cast(hcpcs_modifier_1 as {{ dbt.type_string() }} ) as hcpcs_modifier_1 + , cast(hcpcs_modifier_2 as {{ dbt.type_string() }} ) as hcpcs_modifier_2 + , cast(hcpcs_modifier_3 as {{ dbt.type_string() }} ) as hcpcs_modifier_3 + , cast(hcpcs_modifier_4 as {{ dbt.type_string() }} ) as hcpcs_modifier_4 + , cast(hcpcs_modifier_5 as {{ dbt.type_string() }} ) as hcpcs_modifier_5 + , cast(rendering_npi as {{ dbt.type_string() }} ) as rendering_npi + , cast(rendering_tin as {{ dbt.type_string() }} ) as rendering_tin + , cast(billing_npi as {{ dbt.type_string() }} ) as billing_npi + , cast(billing_tin as {{ dbt.type_string() }} ) as billing_tin + , cast(facility_npi as {{ dbt.type_string() }} ) as facility_npi + , {{ try_to_cast_date('paid_date', 'YYYY-MM-DD') }} as paid_date + , {{ cast_numeric('paid_amount') }} as paid_amount + , {{ cast_numeric('allowed_amount') }} as allowed_amount + , {{ cast_numeric('charge_amount') }} as charge_amount + , {{ cast_numeric('coinsurance_amount') }} as coinsurance_amount + , {{ cast_numeric('copayment_amount') }} as copayment_amount + , {{ cast_numeric('deductible_amount') }} as deductible_amount + , {{ cast_numeric('total_cost_amount') }} as total_cost_amount + , cast(diagnosis_code_type as {{ dbt.type_string() }} ) as diagnosis_code_type + , cast(diagnosis_code_1 as {{ dbt.type_string() }} ) as diagnosis_code_1 + , cast(diagnosis_code_2 as {{ dbt.type_string() }} ) as diagnosis_code_2 + , cast(diagnosis_code_3 as {{ dbt.type_string() }} ) as diagnosis_code_3 + , cast(diagnosis_code_4 as {{ dbt.type_string() }} ) as diagnosis_code_4 + , cast(diagnosis_code_5 as {{ dbt.type_string() }} ) as diagnosis_code_5 + , cast(diagnosis_code_6 as {{ dbt.type_string() }} ) as diagnosis_code_6 + , cast(diagnosis_code_7 as {{ dbt.type_string() }} ) as diagnosis_code_7 + , cast(diagnosis_code_8 as {{ dbt.type_string() }} ) as diagnosis_code_8 + , cast(diagnosis_code_9 as {{ dbt.type_string() }} ) as diagnosis_code_9 + , cast(diagnosis_code_10 as {{ dbt.type_string() }} ) as diagnosis_code_10 + , cast(diagnosis_code_11 as {{ dbt.type_string() }} ) as diagnosis_code_11 + , cast(diagnosis_code_12 as {{ dbt.type_string() }} ) as diagnosis_code_12 + , cast(diagnosis_code_13 as {{ dbt.type_string() }} ) as diagnosis_code_13 + , cast(diagnosis_code_14 as {{ dbt.type_string() }} ) as diagnosis_code_14 + , cast(diagnosis_code_15 as {{ dbt.type_string() }} ) as diagnosis_code_15 + , cast(diagnosis_code_16 as {{ dbt.type_string() }} ) as diagnosis_code_16 + , cast(diagnosis_code_17 as {{ dbt.type_string() }} ) as diagnosis_code_17 + , cast(diagnosis_code_18 as {{ dbt.type_string() }} ) as diagnosis_code_18 + , cast(diagnosis_code_19 as {{ dbt.type_string() }} ) as diagnosis_code_19 + , cast(diagnosis_code_20 as {{ dbt.type_string() }} ) as diagnosis_code_20 + , cast(diagnosis_code_21 as {{ dbt.type_string() }} ) as diagnosis_code_21 + , cast(diagnosis_code_22 as {{ dbt.type_string() }} ) as diagnosis_code_22 + , cast(diagnosis_code_23 as {{ dbt.type_string() }} ) as diagnosis_code_23 + , cast(diagnosis_code_24 as {{ dbt.type_string() }} ) as diagnosis_code_24 + , cast(diagnosis_code_25 as {{ dbt.type_string() }} ) as diagnosis_code_25 + , cast(diagnosis_poa_1 as {{ dbt.type_string() }} ) as diagnosis_poa_1 + , cast(diagnosis_poa_2 as {{ dbt.type_string() }} ) as diagnosis_poa_2 + , cast(diagnosis_poa_3 as {{ dbt.type_string() }} ) as diagnosis_poa_3 + , cast(diagnosis_poa_4 as {{ dbt.type_string() }} ) as diagnosis_poa_4 + , cast(diagnosis_poa_5 as {{ dbt.type_string() }} ) as diagnosis_poa_5 + , cast(diagnosis_poa_6 as {{ dbt.type_string() }} ) as diagnosis_poa_6 + , cast(diagnosis_poa_7 as {{ dbt.type_string() }} ) as diagnosis_poa_7 + , cast(diagnosis_poa_8 as {{ dbt.type_string() }} ) as diagnosis_poa_8 + , cast(diagnosis_poa_9 as {{ dbt.type_string() }} ) as diagnosis_poa_9 + , cast(diagnosis_poa_10 as {{ dbt.type_string() }} ) as diagnosis_poa_10 + , cast(diagnosis_poa_11 as {{ dbt.type_string() }} ) as diagnosis_poa_11 + , cast(diagnosis_poa_12 as {{ dbt.type_string() }} ) as diagnosis_poa_12 + , cast(diagnosis_poa_13 as {{ dbt.type_string() }} ) as diagnosis_poa_13 + , cast(diagnosis_poa_14 as {{ dbt.type_string() }} ) as diagnosis_poa_14 + , cast(diagnosis_poa_15 as {{ dbt.type_string() }} ) as diagnosis_poa_15 + , cast(diagnosis_poa_16 as {{ dbt.type_string() }} ) as diagnosis_poa_16 + , cast(diagnosis_poa_17 as {{ dbt.type_string() }} ) as diagnosis_poa_17 + , cast(diagnosis_poa_18 as {{ dbt.type_string() }} ) as diagnosis_poa_18 + , cast(diagnosis_poa_19 as {{ dbt.type_string() }} ) as diagnosis_poa_19 + , cast(diagnosis_poa_20 as {{ dbt.type_string() }} ) as diagnosis_poa_20 + , cast(diagnosis_poa_21 as {{ dbt.type_string() }} ) as diagnosis_poa_21 + , cast(diagnosis_poa_22 as {{ dbt.type_string() }} ) as diagnosis_poa_22 + , cast(diagnosis_poa_23 as {{ dbt.type_string() }} ) as diagnosis_poa_23 + , cast(diagnosis_poa_24 as {{ dbt.type_string() }} ) as diagnosis_poa_24 + , cast(diagnosis_poa_25 as {{ dbt.type_string() }} ) as diagnosis_poa_25 + , cast(procedure_code_type as {{ dbt.type_string() }} ) as procedure_code_type + , cast(procedure_code_1 as {{ dbt.type_string() }} ) as procedure_code_1 + , cast(procedure_code_2 as {{ dbt.type_string() }} ) as procedure_code_2 + , cast(procedure_code_3 as {{ dbt.type_string() }} ) as procedure_code_3 + , cast(procedure_code_4 as {{ dbt.type_string() }} ) as procedure_code_4 + , cast(procedure_code_5 as {{ dbt.type_string() }} ) as procedure_code_5 + , cast(procedure_code_6 as {{ dbt.type_string() }} ) as procedure_code_6 + , cast(procedure_code_7 as {{ dbt.type_string() }} ) as procedure_code_7 + , cast(procedure_code_8 as {{ dbt.type_string() }} ) as procedure_code_8 + , cast(procedure_code_9 as {{ dbt.type_string() }} ) as procedure_code_9 + , cast(procedure_code_10 as {{ dbt.type_string() }} ) as procedure_code_10 + , cast(procedure_code_11 as {{ dbt.type_string() }} ) as procedure_code_11 + , cast(procedure_code_12 as {{ dbt.type_string() }} ) as procedure_code_12 + , cast(procedure_code_13 as {{ dbt.type_string() }} ) as procedure_code_13 + , cast(procedure_code_14 as {{ dbt.type_string() }} ) as procedure_code_14 + , cast(procedure_code_15 as {{ dbt.type_string() }} ) as procedure_code_15 + , cast(procedure_code_16 as {{ dbt.type_string() }} ) as procedure_code_16 + , cast(procedure_code_17 as {{ dbt.type_string() }} ) as procedure_code_17 + , cast(procedure_code_18 as {{ dbt.type_string() }} ) as procedure_code_18 + , cast(procedure_code_19 as {{ dbt.type_string() }} ) as procedure_code_19 + , cast(procedure_code_20 as {{ dbt.type_string() }} ) as procedure_code_20 + , cast(procedure_code_21 as {{ dbt.type_string() }} ) as procedure_code_21 + , cast(procedure_code_22 as {{ dbt.type_string() }} ) as procedure_code_22 + , cast(procedure_code_23 as {{ dbt.type_string() }} ) as procedure_code_23 + , cast(procedure_code_24 as {{ dbt.type_string() }} ) as procedure_code_24 + , cast(procedure_code_25 as {{ dbt.type_string() }} ) as procedure_code_25 + , {{ try_to_cast_date('procedure_date_1', 'YYYY-MM-DD') }} as procedure_date_1 + , {{ try_to_cast_date('procedure_date_2', 'YYYY-MM-DD') }} as procedure_date_2 + , {{ try_to_cast_date('procedure_date_3', 'YYYY-MM-DD') }} as procedure_date_3 + , {{ try_to_cast_date('procedure_date_4', 'YYYY-MM-DD') }} as procedure_date_4 + , {{ try_to_cast_date('procedure_date_5', 'YYYY-MM-DD') }} as procedure_date_5 + , {{ try_to_cast_date('procedure_date_6', 'YYYY-MM-DD') }} as procedure_date_6 + , {{ try_to_cast_date('procedure_date_7', 'YYYY-MM-DD') }} as procedure_date_7 + , {{ try_to_cast_date('procedure_date_8', 'YYYY-MM-DD') }} as procedure_date_8 + , {{ try_to_cast_date('procedure_date_9', 'YYYY-MM-DD') }} as procedure_date_9 + , {{ try_to_cast_date('procedure_date_10', 'YYYY-MM-DD') }} as procedure_date_10 + , {{ try_to_cast_date('procedure_date_11', 'YYYY-MM-DD') }} as procedure_date_11 + , {{ try_to_cast_date('procedure_date_12', 'YYYY-MM-DD') }} as procedure_date_12 + , {{ try_to_cast_date('procedure_date_13', 'YYYY-MM-DD') }} as procedure_date_13 + , {{ try_to_cast_date('procedure_date_14', 'YYYY-MM-DD') }} as procedure_date_14 + , {{ try_to_cast_date('procedure_date_15', 'YYYY-MM-DD') }} as procedure_date_15 + , {{ try_to_cast_date('procedure_date_16', 'YYYY-MM-DD') }} as procedure_date_16 + , {{ try_to_cast_date('procedure_date_17', 'YYYY-MM-DD') }} as procedure_date_17 + , {{ try_to_cast_date('procedure_date_18', 'YYYY-MM-DD') }} as procedure_date_18 + , {{ try_to_cast_date('procedure_date_19', 'YYYY-MM-DD') }} as procedure_date_19 + , {{ try_to_cast_date('procedure_date_20', 'YYYY-MM-DD') }} as procedure_date_20 + , {{ try_to_cast_date('procedure_date_21', 'YYYY-MM-DD') }} as procedure_date_21 + , {{ try_to_cast_date('procedure_date_22', 'YYYY-MM-DD') }} as procedure_date_22 + , {{ try_to_cast_date('procedure_date_23', 'YYYY-MM-DD') }} as procedure_date_23 + , {{ try_to_cast_date('procedure_date_24', 'YYYY-MM-DD') }} as procedure_date_24 + , {{ try_to_cast_date('procedure_date_25', 'YYYY-MM-DD') }} as procedure_date_25 + , cast(in_network_flag as integer) as in_network_flag + , cast(data_source as {{ dbt.type_string() }} ) as data_source + , cast(file_name as {{ dbt.type_string() }} ) as file_name + , cast(ingest_datetime as {{ dbt.type_string() }} ) as ingest_datetime + from mapping + +) + +select + claim_id + , claim_line_number + , claim_type + , patient_id + , member_id + , payer + , {{ the_tuva_project.quote_column('plan') }} + , claim_start_date + , claim_end_date + , claim_line_start_date + , claim_line_end_date + , admission_date + , discharge_date + , admit_source_code + , admit_type_code + , discharge_disposition_code + , place_of_service_code + , bill_type_code + , ms_drg_code + , apr_drg_code + , revenue_center_code + , service_unit_quantity + , hcpcs_code + , hcpcs_modifier_1 + , hcpcs_modifier_2 + , hcpcs_modifier_3 + , hcpcs_modifier_4 + , hcpcs_modifier_5 + , rendering_npi + , rendering_tin + , billing_npi + , billing_tin + , facility_npi + , paid_date + , paid_amount + , allowed_amount + , charge_amount + , coinsurance_amount + , copayment_amount + , deductible_amount + , total_cost_amount + , diagnosis_code_type + , diagnosis_code_1 + , diagnosis_code_2 + , diagnosis_code_3 + , diagnosis_code_4 + , diagnosis_code_5 + , diagnosis_code_6 + , diagnosis_code_7 + , diagnosis_code_8 + , diagnosis_code_9 + , diagnosis_code_10 + , diagnosis_code_11 + , diagnosis_code_12 + , diagnosis_code_13 + , diagnosis_code_14 + , diagnosis_code_15 + , diagnosis_code_16 + , diagnosis_code_17 + , diagnosis_code_18 + , diagnosis_code_19 + , diagnosis_code_20 + , diagnosis_code_21 + , diagnosis_code_22 + , diagnosis_code_23 + , diagnosis_code_24 + , diagnosis_code_25 + , diagnosis_poa_1 + , diagnosis_poa_2 + , diagnosis_poa_3 + , diagnosis_poa_4 + , diagnosis_poa_5 + , diagnosis_poa_6 + , diagnosis_poa_7 + , diagnosis_poa_8 + , diagnosis_poa_9 + , diagnosis_poa_10 + , diagnosis_poa_11 + , diagnosis_poa_12 + , diagnosis_poa_13 + , diagnosis_poa_14 + , diagnosis_poa_15 + , diagnosis_poa_16 + , diagnosis_poa_17 + , diagnosis_poa_18 + , diagnosis_poa_19 + , diagnosis_poa_20 + , diagnosis_poa_21 + , diagnosis_poa_22 + , diagnosis_poa_23 + , diagnosis_poa_24 + , diagnosis_poa_25 + , procedure_code_type + , procedure_code_1 + , procedure_code_2 + , procedure_code_3 + , procedure_code_4 + , procedure_code_5 + , procedure_code_6 + , procedure_code_7 + , procedure_code_8 + , procedure_code_9 + , procedure_code_10 + , procedure_code_11 + , procedure_code_12 + , procedure_code_13 + , procedure_code_14 + , procedure_code_15 + , procedure_code_16 + , procedure_code_17 + , procedure_code_18 + , procedure_code_19 + , procedure_code_20 + , procedure_code_21 + , procedure_code_22 + , procedure_code_23 + , procedure_code_24 + , procedure_code_25 + , procedure_date_1 + , procedure_date_2 + , procedure_date_3 + , procedure_date_4 + , procedure_date_5 + , procedure_date_6 + , procedure_date_7 + , procedure_date_8 + , procedure_date_9 + , procedure_date_10 + , procedure_date_11 + , procedure_date_12 + , procedure_date_13 + , procedure_date_14 + , procedure_date_15 + , procedure_date_16 + , procedure_date_17 + , procedure_date_18 + , procedure_date_19 + , procedure_date_20 + , procedure_date_21 + , procedure_date_22 + , procedure_date_23 + , procedure_date_24 + , procedure_date_25 + , in_network_flag + , data_source + , file_name + , ingest_datetime +from add_data_types \ No newline at end of file diff --git a/models/intermediate/int_procedure_deduped.sql b/models/intermediate/int_procedure_deduped.sql new file mode 100644 index 00000000..d2d85ee0 --- /dev/null +++ b/models/intermediate/int_procedure_deduped.sql @@ -0,0 +1,63 @@ +with staged_data as ( + + select + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_val_sqnc_num + , clm_prcdr_cd + , clm_prcdr_prfrm_dt + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , dgns_prcdr_icd_ind + , file_name + , file_date + from {{ ref('stg_parta_procedure_code') }} + +) + +/* dedupe full rows that may appear in multiple files */ +, add_row_num as ( + + select *, row_number() over ( + partition by + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_val_sqnc_num + , clm_prcdr_cd + , clm_prcdr_prfrm_dt + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , dgns_prcdr_icd_ind + order by file_date desc + ) as row_num + from staged_data + where bene_mbi_id is not null /* added to prevent dupes during pivot */ + +) + +/* cast data types before pivot operation */ +select + cast(cur_clm_uniq_id as {{ dbt.type_string() }} ) as cur_clm_uniq_id + , cast(bene_mbi_id as {{ dbt.type_string() }} ) as bene_mbi_id + , cast(bene_hic_num as {{ dbt.type_string() }} ) as bene_hic_num + , cast(clm_type_cd as {{ dbt.type_string() }} ) as clm_type_cd + , cast(cast(clm_val_sqnc_num as integer) as {{ dbt.type_string() }} ) as clm_val_sqnc_num + , cast(clm_prcdr_cd as {{ dbt.type_string() }} ) as clm_prcdr_cd + , cast(clm_prcdr_prfrm_dt as {{ dbt.type_string() }} ) as clm_prcdr_prfrm_dt + , cast(bene_eqtbl_bic_hicn_num as {{ dbt.type_string() }} ) as bene_eqtbl_bic_hicn_num + , cast(prvdr_oscar_num as {{ dbt.type_string() }} ) as prvdr_oscar_num + , cast(clm_from_dt as {{ dbt.type_string() }} ) as clm_from_dt + , cast(clm_thru_dt as {{ dbt.type_string() }} ) as clm_thru_dt + , cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) as dgns_prcdr_icd_ind + , file_name + , file_date +from add_row_num +where row_num = 1 \ No newline at end of file diff --git a/models/intermediate/procedure_pivot.sql b/models/intermediate/int_procedure_pivot.sql similarity index 89% rename from models/intermediate/procedure_pivot.sql rename to models/intermediate/int_procedure_pivot.sql index c56b5c28..2fff2dd6 100644 --- a/models/intermediate/procedure_pivot.sql +++ b/models/intermediate/int_procedure_pivot.sql @@ -1,105 +1,103 @@ -with procedure_pivot as ( - - select - cur_clm_uniq_id - , bene_mbi_id - , dgns_prcdr_icd_ind - , {{ dbt_utils.pivot( - column='clm_val_sqnc_num' - , values=['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25'] - , agg='max' - , then_value='clm_prcdr_cd' - , else_value='null' - , prefix='procedure_code_' - , quote_identifiers=false - ) }} - from {{ ref('stg_parta_procedure_code') }} - group by - cur_clm_uniq_id - , bene_mbi_id - , dgns_prcdr_icd_ind - -), - -date_pivot as( - - select - cur_clm_uniq_id - , bene_mbi_id - , dgns_prcdr_icd_ind - , {{ dbt_utils.pivot( - column='clm_val_sqnc_num' - , values=['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25'] - , agg='max' - , then_value='clm_prcdr_prfrm_dt' - , else_value='null' - , prefix='procedure_date_' - , quote_identifiers=false - ) }} - from {{ ref('stg_parta_procedure_code') }} - group by - cur_clm_uniq_id - , bene_mbi_id - , dgns_prcdr_icd_ind - -) - -select - px.cur_clm_uniq_id - , px.bene_mbi_id - , px.dgns_prcdr_icd_ind - , px.procedure_code_1 - , px.procedure_code_2 - , px.procedure_code_3 - , px.procedure_code_4 - , px.procedure_code_5 - , px.procedure_code_6 - , px.procedure_code_7 - , px.procedure_code_8 - , px.procedure_code_9 - , px.procedure_code_10 - , px.procedure_code_11 - , px.procedure_code_12 - , px.procedure_code_13 - , px.procedure_code_14 - , px.procedure_code_15 - , px.procedure_code_16 - , px.procedure_code_17 - , px.procedure_code_18 - , px.procedure_code_19 - , px.procedure_code_20 - , px.procedure_code_21 - , px.procedure_code_22 - , px.procedure_code_23 - , px.procedure_code_24 - , px.procedure_code_25 - , d.procedure_date_1 - , d.procedure_date_2 - , d.procedure_date_3 - , d.procedure_date_4 - , d.procedure_date_5 - , d.procedure_date_6 - , d.procedure_date_7 - , d.procedure_date_8 - , d.procedure_date_9 - , d.procedure_date_10 - , d.procedure_date_11 - , d.procedure_date_12 - , d.procedure_date_13 - , d.procedure_date_14 - , d.procedure_date_15 - , d.procedure_date_16 - , d.procedure_date_17 - , d.procedure_date_18 - , d.procedure_date_19 - , d.procedure_date_20 - , d.procedure_date_21 - , d.procedure_date_22 - , d.procedure_date_23 - , d.procedure_date_24 - , d.procedure_date_25 -from procedure_pivot as px - inner join date_pivot as d - on px.cur_clm_uniq_id = d.cur_clm_uniq_id -/* filtering out null values from seed file */ -where px.cur_clm_uniq_id is not null \ No newline at end of file +with procedure_pivot as ( + + select + cur_clm_uniq_id + , bene_mbi_id + , dgns_prcdr_icd_ind + , {{ dbt_utils.pivot( + column='clm_val_sqnc_num' + , values=['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25'] + , agg='max' + , then_value='clm_prcdr_cd' + , else_value='null' + , prefix='procedure_code_' + , quote_identifiers=false + ) }} + from {{ ref('int_procedure_deduped') }} + group by + cur_clm_uniq_id + , bene_mbi_id + , dgns_prcdr_icd_ind + +), + +date_pivot as( + + select + cur_clm_uniq_id + , bene_mbi_id + , dgns_prcdr_icd_ind + , {{ dbt_utils.pivot( + column='clm_val_sqnc_num' + , values=['1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25'] + , agg='max' + , then_value='clm_prcdr_prfrm_dt' + , else_value='null' + , prefix='procedure_date_' + , quote_identifiers=false + ) }} + from {{ ref('int_procedure_deduped') }} + group by + cur_clm_uniq_id + , bene_mbi_id + , dgns_prcdr_icd_ind + +) + +select + px.cur_clm_uniq_id + , px.bene_mbi_id + , px.dgns_prcdr_icd_ind + , px.procedure_code_1 + , px.procedure_code_2 + , px.procedure_code_3 + , px.procedure_code_4 + , px.procedure_code_5 + , px.procedure_code_6 + , px.procedure_code_7 + , px.procedure_code_8 + , px.procedure_code_9 + , px.procedure_code_10 + , px.procedure_code_11 + , px.procedure_code_12 + , px.procedure_code_13 + , px.procedure_code_14 + , px.procedure_code_15 + , px.procedure_code_16 + , px.procedure_code_17 + , px.procedure_code_18 + , px.procedure_code_19 + , px.procedure_code_20 + , px.procedure_code_21 + , px.procedure_code_22 + , px.procedure_code_23 + , px.procedure_code_24 + , px.procedure_code_25 + , d.procedure_date_1 + , d.procedure_date_2 + , d.procedure_date_3 + , d.procedure_date_4 + , d.procedure_date_5 + , d.procedure_date_6 + , d.procedure_date_7 + , d.procedure_date_8 + , d.procedure_date_9 + , d.procedure_date_10 + , d.procedure_date_11 + , d.procedure_date_12 + , d.procedure_date_13 + , d.procedure_date_14 + , d.procedure_date_15 + , d.procedure_date_16 + , d.procedure_date_17 + , d.procedure_date_18 + , d.procedure_date_19 + , d.procedure_date_20 + , d.procedure_date_21 + , d.procedure_date_22 + , d.procedure_date_23 + , d.procedure_date_24 + , d.procedure_date_25 +from procedure_pivot as px + inner join date_pivot as d + on px.cur_clm_uniq_id = d.cur_clm_uniq_id \ No newline at end of file diff --git a/models/intermediate/int_revenue_center_deduped.sql b/models/intermediate/int_revenue_center_deduped.sql new file mode 100644 index 00000000..627079d0 --- /dev/null +++ b/models/intermediate/int_revenue_center_deduped.sql @@ -0,0 +1,162 @@ +with staged_data as ( + + select + cur_clm_uniq_id + , clm_line_num + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_prod_rev_ctr_cd + , clm_line_instnl_rev_ctr_dt + , clm_line_hcpcs_cd + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , clm_line_srvc_unit_qty + , clm_line_cvrd_pd_amt + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_rev_apc_hipps_cd + , file_name + , file_date + from {{ ref('stg_parta_claims_revenue_center_detail') }} + +) + +, beneficiary_xref as ( + + select * from {{ ref('int_beneficiary_xref_deduped') }} + +) + +/* coalesce current MBI from XREF if exists and MBI on claim */ +, add_current_mbi as ( + + select + cur_clm_uniq_id + , clm_line_num + , coalesce(beneficiary_xref.crnt_num, staged_data.bene_mbi_id) as current_bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_prod_rev_ctr_cd + , clm_line_instnl_rev_ctr_dt + , clm_line_hcpcs_cd + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , clm_line_srvc_unit_qty + , clm_line_cvrd_pd_amt + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_rev_apc_hipps_cd + , file_name + , file_date + from staged_data + left join beneficiary_xref + on staged_data.bene_mbi_id = beneficiary_xref.prvs_num + +) + +/* dedupe full rows that may appear in multiple files */ +, add_row_num as ( + + select *, row_number() over ( + partition by + cur_clm_uniq_id + , clm_line_num + , current_bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_prod_rev_ctr_cd + , clm_line_instnl_rev_ctr_dt + , clm_line_hcpcs_cd + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , clm_line_srvc_unit_qty + , clm_line_cvrd_pd_amt + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_rev_apc_hipps_cd + order by file_date desc + ) as row_num + from add_current_mbi + +) + +, dedupe as ( + + select + cur_clm_uniq_id + , clm_line_num + , current_bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_prod_rev_ctr_cd + , clm_line_instnl_rev_ctr_dt + , clm_line_hcpcs_cd + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , clm_line_srvc_unit_qty + , clm_line_cvrd_pd_amt + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_rev_apc_hipps_cd + , file_name + , file_date + from add_row_num + where row_num = 1 + +) + +select distinct + cur_clm_uniq_id + , clm_line_num + , current_bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_line_from_dt + , clm_line_thru_dt + , clm_line_prod_rev_ctr_cd + , clm_line_instnl_rev_ctr_dt + , clm_line_hcpcs_cd + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , clm_line_srvc_unit_qty + , clm_line_cvrd_pd_amt + , hcpcs_1_mdfr_cd + , hcpcs_2_mdfr_cd + , hcpcs_3_mdfr_cd + , hcpcs_4_mdfr_cd + , hcpcs_5_mdfr_cd + , clm_rev_apc_hipps_cd + , file_name + , file_date +from dedupe \ No newline at end of file diff --git a/models/intermediate/physician_claims.sql b/models/intermediate/physician_claims.sql deleted file mode 100644 index d18c63e7..00000000 --- a/models/intermediate/physician_claims.sql +++ /dev/null @@ -1,155 +0,0 @@ -select - cast(cur_clm_uniq_id as {{ dbt.type_string() }} ) as claim_id - , cast(clm_line_num as integer) as claim_line_number - , 'professional' as claim_type - , cast(bene_mbi_id as {{ dbt.type_string() }} ) as patient_id - , cast(bene_mbi_id as {{ dbt.type_string() }} ) as member_id - , 'medicare' as payer - , 'medicare' as plan - , {{ try_to_cast_date('clm_from_dt', 'YYYY-MM-DD') }} as claim_start_date - , {{ try_to_cast_date('clm_thru_dt', 'YYYY-MM-DD') }} as claim_end_date - , {{ try_to_cast_date('clm_line_from_dt', 'YYYY-MM-DD') }} as claim_line_start_date - , {{ try_to_cast_date('clm_line_thru_dt', 'YYYY-MM-DD') }} as claim_line_end_date - , cast(NULL as date) as admission_date - , cast(NULL as date) as discharge_date - , cast(NULL as {{ dbt.type_string() }} ) as admit_source_code - , cast(NULL as {{ dbt.type_string() }} ) as admit_type_code - , cast(NULL as {{ dbt.type_string() }} ) as discharge_disposition_code - , cast(clm_pos_cd as {{ dbt.type_string() }} ) as place_of_service_code - , cast(NULL as {{ dbt.type_string() }} ) as bill_type_code - , cast(NULL as {{ dbt.type_string() }} ) as ms_drg_code - , cast(NULL as {{ dbt.type_string() }} ) as apr_drg_code - , cast(NULL as {{ dbt.type_string() }} ) as revenue_center_code - , cast(clm_line_srvc_unit_qty as integer) as service_unit_quantity - , cast(clm_line_hcpcs_cd as {{ dbt.type_string() }} ) as hcpcs_code - , cast(hcpcs_1_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_1 - , cast(hcpcs_2_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_2 - , cast(hcpcs_3_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_3 - , cast(hcpcs_4_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_4 - , cast(hcpcs_5_mdfr_cd as {{ dbt.type_string() }} ) as hcpcs_modifier_5 - , cast(rndrg_prvdr_npi_num as {{ dbt.type_string() }} ) as rendering_npi - , cast(clm_rndrg_prvdr_tax_num as {{ dbt.type_string() }} ) as rendering_tin - , cast(NULL as {{ dbt.type_string() }} ) as billing_npi - , cast(NULL as {{ dbt.type_string() }} ) as billing_tin - , cast(NULL as {{ dbt.type_string() }} ) as facility_npi - , cast(NULL as date) as paid_date - , case - when cast(clm_adjsmt_type_cd as {{ dbt.type_string() }} ) = '1' then {{ cast_numeric('clm_line_cvrd_pd_amt') }} * -1 - else {{ cast_numeric('clm_line_cvrd_pd_amt') }} - end as paid_amount - , {{ cast_numeric('clm_line_alowd_chrg_amt') }} as allowed_amount - , {{ cast_numeric('clm_line_alowd_chrg_amt') }} as charge_amount - , {{ cast_numeric('NULL') }} as coinsurance_amount - , {{ cast_numeric('NULL') }} as copayment_amount - , {{ cast_numeric('NULL') }} as deductible_amount - , {{ cast_numeric('NULL') }} as total_cost_amount - , case - when cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) = '0' then 'icd-10-cm' - when cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) = '9' then 'icd-9-cm' - else cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) end as diagnosis_code_type - , cast(clm_dgns_1_cd as {{ dbt.type_string() }} ) as diagnosis_code_1 - , cast(clm_dgns_2_cd as {{ dbt.type_string() }} ) as diagnosis_code_2 - , cast(clm_dgns_3_cd as {{ dbt.type_string() }} ) as diagnosis_code_3 - , cast(clm_dgns_4_cd as {{ dbt.type_string() }} ) as diagnosis_code_4 - , cast(clm_dgns_5_cd as {{ dbt.type_string() }} ) as diagnosis_code_5 - , cast(clm_dgns_6_cd as {{ dbt.type_string() }} ) as diagnosis_code_6 - , cast(clm_dgns_7_cd as {{ dbt.type_string() }} ) as diagnosis_code_7 - , cast(clm_dgns_8_cd as {{ dbt.type_string() }} ) as diagnosis_code_8 - , cast(clm_dgns_9_cd as {{ dbt.type_string() }} ) as diagnosis_code_9 - , cast(clm_dgns_10_cd as {{ dbt.type_string() }} ) as diagnosis_code_10 - , cast(clm_dgns_11_cd as {{ dbt.type_string() }} ) as diagnosis_code_11 - , cast(clm_dgns_12_cd as {{ dbt.type_string() }} ) as diagnosis_code_12 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_13 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_14 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_15 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_16 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_17 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_18 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_19 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_20 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_21 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_22 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_23 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_24 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_code_25 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_1 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_2 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_3 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_4 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_5 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_6 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_7 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_8 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_9 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_10 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_11 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_12 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_13 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_14 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_15 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_16 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_17 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_18 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_19 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_20 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_21 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_22 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_23 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_24 - , cast(NULL as {{ dbt.type_string() }} ) as diagnosis_poa_25 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_type - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_1 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_2 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_3 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_4 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_5 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_6 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_7 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_8 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_9 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_10 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_11 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_12 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_13 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_14 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_15 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_16 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_17 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_18 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_19 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_20 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_21 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_22 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_23 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_24 - , cast(NULL as {{ dbt.type_string() }} ) as procedure_code_25 - , cast(NULL as date) as procedure_date_1 - , cast(NULL as date) as procedure_date_2 - , cast(NULL as date) as procedure_date_3 - , cast(NULL as date) as procedure_date_4 - , cast(NULL as date) as procedure_date_5 - , cast(NULL as date) as procedure_date_6 - , cast(NULL as date) as procedure_date_7 - , cast(NULL as date) as procedure_date_8 - , cast(NULL as date) as procedure_date_9 - , cast(NULL as date) as procedure_date_10 - , cast(NULL as date) as procedure_date_11 - , cast(NULL as date) as procedure_date_12 - , cast(NULL as date) as procedure_date_13 - , cast(NULL as date) as procedure_date_14 - , cast(NULL as date) as procedure_date_15 - , cast(NULL as date) as procedure_date_16 - , cast(NULL as date) as procedure_date_17 - , cast(NULL as date) as procedure_date_18 - , cast(NULL as date) as procedure_date_19 - , cast(NULL as date) as procedure_date_20 - , cast(NULL as date) as procedure_date_21 - , cast(NULL as date) as procedure_date_22 - , cast(NULL as date) as procedure_date_23 - , cast(NULL as date) as procedure_date_24 - , cast(NULL as date) as procedure_date_25 - , cast(1 as int) as in_network_flag - , 'medicare cclf' as data_source - , cast(file_name as {{ dbt.type_string() }} ) as file_name - , cast(ingest_datetime as {{ dbt.type_timestamp() }} ) as ingest_datetime -from {{ ref('stg_partb_physicians') }} \ No newline at end of file diff --git a/models/staging/stg_beneficiary_demographics.sql b/models/staging/stg_beneficiary_demographics.sql index f1d825fa..f76d33e4 100644 --- a/models/staging/stg_beneficiary_demographics.sql +++ b/models/staging/stg_beneficiary_demographics.sql @@ -1,6 +1,5 @@ select bene_mbi_id - , bene_member_month , bene_hic_num , bene_fips_state_cd , bene_fips_cnty_cd @@ -32,5 +31,5 @@ select , geo_zip5_cd , geo_zip4_cd , file_name - , ingest_datetime + , file_date from {{ source('medicare_cclf','beneficiary_demographics') }} \ No newline at end of file diff --git a/models/staging/stg_beneficiary_xref.sql b/models/staging/stg_beneficiary_xref.sql new file mode 100644 index 00000000..ac4e9e61 --- /dev/null +++ b/models/staging/stg_beneficiary_xref.sql @@ -0,0 +1,10 @@ +select + hicn_mbi_xref_ind + , crnt_num + , prvs_num + , prvs_id_efctv_dt + , prvs_id_obslt_dt + , bene_rrb_num + , file_name + , file_date +from {{ source('medicare_cclf','beneficiary_xref') }} \ No newline at end of file diff --git a/models/staging/stg_enrollment.sql b/models/staging/stg_enrollment.sql new file mode 100644 index 00000000..67a734d9 --- /dev/null +++ b/models/staging/stg_enrollment.sql @@ -0,0 +1,8 @@ +select + current_bene_mbi_id + , enrollment_start_date + , enrollment_end_date + , bene_member_month + , file_name + , file_date +from {{ source('medicare_cclf','enrollment') }} \ No newline at end of file diff --git a/models/staging/stg_parta_claims_header.sql b/models/staging/stg_parta_claims_header.sql index 0ed48c93..5c6211e4 100644 --- a/models/staging/stg_parta_claims_header.sql +++ b/models/staging/stg_parta_claims_header.sql @@ -36,6 +36,14 @@ select , clm_mdcr_ip_pps_dsprprtnt_amt , clm_hipps_uncompd_care_amt , clm_oprtnl_dsprprtnt_amt + , clm_blg_prvdr_oscar_num + , clm_blg_prvdr_npi_num + , clm_oprtg_prvdr_npi_num + , clm_atndg_prvdr_npi_num + , clm_othr_prvdr_npi_num + , clm_cntl_num + , clm_org_cntl_num + , clm_cntrctr_num , file_name - , ingest_datetime + , file_date from {{ source('medicare_cclf','parta_claims_header') }} diff --git a/models/staging/stg_parta_claims_revenue_center_detail.sql b/models/staging/stg_parta_claims_revenue_center_detail.sql index 5ffd9afe..edfa7116 100644 --- a/models/staging/stg_parta_claims_revenue_center_detail.sql +++ b/models/staging/stg_parta_claims_revenue_center_detail.sql @@ -22,5 +22,5 @@ select , hcpcs_5_mdfr_cd , clm_rev_apc_hipps_cd , file_name - , ingest_datetime + , file_date from {{ source('medicare_cclf','parta_claims_revenue_center_detail') }} \ No newline at end of file diff --git a/models/staging/stg_parta_diagnosis_code.sql b/models/staging/stg_parta_diagnosis_code.sql index b79d7160..4a2bb8b4 100644 --- a/models/staging/stg_parta_diagnosis_code.sql +++ b/models/staging/stg_parta_diagnosis_code.sql @@ -1,15 +1,17 @@ select - cast(cur_clm_uniq_id as {{ dbt.type_string() }} ) as cur_clm_uniq_id - , cast(bene_mbi_id as {{ dbt.type_string() }} ) as bene_mbi_id - , cast(bene_hic_num as {{ dbt.type_string() }} ) as bene_hic_num - , cast(clm_type_cd as {{ dbt.type_string() }} ) as clm_type_cd - , cast(clm_prod_type_cd as {{ dbt.type_string() }} ) as clm_prod_type_cd - , cast(clm_val_sqnc_num as {{ dbt.type_string() }} ) as clm_val_sqnc_num - , cast(clm_dgns_cd as {{ dbt.type_string() }} ) as clm_dgns_cd - , cast(bene_eqtbl_bic_hicn_num as {{ dbt.type_string() }} ) as bene_eqtbl_bic_hicn_num - , cast(prvdr_oscar_num as {{ dbt.type_string() }} ) as prvdr_oscar_num - , cast(clm_from_dt as {{ dbt.type_string() }} ) as clm_from_dt - , cast(clm_thru_dt as {{ dbt.type_string() }} ) as clm_thru_dt - , cast(clm_poa_ind as {{ dbt.type_string() }} ) as clm_poa_ind - , cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) as dgns_prcdr_icd_ind + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_prod_type_cd + , clm_val_sqnc_num + , clm_dgns_cd + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , clm_poa_ind + , dgns_prcdr_icd_ind + , file_name + , file_date from {{ source('medicare_cclf','parta_diagnosis_code') }} \ No newline at end of file diff --git a/models/staging/stg_parta_procedure_code.sql b/models/staging/stg_parta_procedure_code.sql index 487fafea..57d71da6 100644 --- a/models/staging/stg_parta_procedure_code.sql +++ b/models/staging/stg_parta_procedure_code.sql @@ -1,14 +1,16 @@ select - cast(cur_clm_uniq_id as {{ dbt.type_string() }} ) as cur_clm_uniq_id - , cast(bene_mbi_id as {{ dbt.type_string() }} ) as bene_mbi_id - , cast(bene_hic_num as {{ dbt.type_string() }} ) as bene_hic_num - , cast(clm_type_cd as {{ dbt.type_string() }} ) as clm_type_cd - , cast(clm_val_sqnc_num as {{ dbt.type_string() }} ) as clm_val_sqnc_num - , cast(clm_prcdr_cd as {{ dbt.type_string() }} ) as clm_prcdr_cd - , cast(clm_prcdr_prfrm_dt as {{ dbt.type_string() }} ) as clm_prcdr_prfrm_dt - , cast(bene_eqtbl_bic_hicn_num as {{ dbt.type_string() }} ) as bene_eqtbl_bic_hicn_num - , cast(prvdr_oscar_num as {{ dbt.type_string() }} ) as prvdr_oscar_num - , cast(clm_from_dt as {{ dbt.type_string() }} ) as clm_from_dt - , cast(clm_thru_dt as {{ dbt.type_string() }} ) as clm_thru_dt - , cast(dgns_prcdr_icd_ind as {{ dbt.type_string() }} ) as dgns_prcdr_icd_ind + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_type_cd + , clm_val_sqnc_num + , clm_prcdr_cd + , clm_prcdr_prfrm_dt + , bene_eqtbl_bic_hicn_num + , prvdr_oscar_num + , clm_from_dt + , clm_thru_dt + , dgns_prcdr_icd_ind + , file_name + , file_date from {{ source('medicare_cclf','parta_procedure_code') }} \ No newline at end of file diff --git a/models/staging/stg_partb_dme.sql b/models/staging/stg_partb_dme.sql index 12c62814..116d997d 100644 --- a/models/staging/stg_partb_dme.sql +++ b/models/staging/stg_partb_dme.sql @@ -25,5 +25,5 @@ select , clm_line_alowd_chrg_amt , clm_disp_cd , file_name - , ingest_datetime + , file_date from {{ source('medicare_cclf','partb_dme') }} \ No newline at end of file diff --git a/models/staging/stg_partb_physicians.sql b/models/staging/stg_partb_physicians.sql index 313a7978..14185249 100644 --- a/models/staging/stg_partb_physicians.sql +++ b/models/staging/stg_partb_physicians.sql @@ -49,5 +49,5 @@ select , clm_dgns_12_cd , hcpcs_betos_cd , file_name - , ingest_datetime + , file_date from {{ source('medicare_cclf','partb_physicians') }} \ No newline at end of file diff --git a/models/staging/stg_partd_claims.sql b/models/staging/stg_partd_claims.sql new file mode 100644 index 00000000..f5b52567 --- /dev/null +++ b/models/staging/stg_partd_claims.sql @@ -0,0 +1,25 @@ +select + cur_clm_uniq_id + , bene_mbi_id + , bene_hic_num + , clm_line_ndc_cd + , clm_type_cd + , clm_line_from_dt + , prvdr_srvc_id_qlfyr_cd + , clm_srvc_prvdr_gnrc_id_num + , clm_dspnsng_stus_cd + , clm_daw_prod_slctn_cd + , clm_line_srvc_unit_qty + , clm_line_days_suply_qty + , prvdr_prsbng_id_qlfyr_cd + , clm_prsbng_prvdr_gnrc_id_num + , clm_line_bene_pmt_amt + , clm_adjsmt_type_cd + , clm_efctv_dt + , clm_idr_ld_dt + , clm_line_rx_srvc_rfrnc_num + , clm_line_rx_fill_num + , clm_phrmcy_srvc_type_cd + , file_name + , file_date +from {{ source('medicare_cclf','partd_claims') }} \ No newline at end of file diff --git a/packages.yml b/packages.yml index 1a42c0a1..610b03c6 100644 --- a/packages.yml +++ b/packages.yml @@ -2,4 +2,4 @@ packages: - package: dbt-labs/dbt_utils version: [">=0.9.2","<1.0.0"] - package: tuva-health/the_tuva_project - version: [">=0.10.0","<0.11.0"] \ No newline at end of file + version: [">=0.12.0","<0.13.0"] \ No newline at end of file