Overwriting rendered column descriptions with the unrendered yaml

[VDFaller]

Fixes #128

Obviously this isn't the final but I got this working if I move the docs block from orders to stg_orders. in the demo_sqlite jaffle_shop. Still unclear what patch_path really is so don't know if that's a great idea.

Just making sure I'm not barking up the wrong tree.

Currently looking for a function in dbt core to just get the relevant dict from the fqn or unique_id or something like that.

[z3z1ma]

@VDFaller patch_path is what dbt for years has referred to the the YAML metadata as on the inside of its lib. They call it that because they generate nodes prior to "patching" in the yaml metadata AFAIK. So totally the right thing to use to find the complementary YAML for a model/seed node.

My only thought here is to make sure we know for certain what dbt versions the api your consuming is compatible with so we can advertise it to users. But directionally this looks right to me and that function that constructs the dict with the description key is exactly where id put this.

[z3z1ma]

LMK if you are comfortable with the click part, otherwise I can help run this to the finish line. Up to you.

[VDFaller]

@z3z1ma is your testing setup for multiple dbt versions it you just want me to manually test? Or just look at manifest versions ?

I'll take a stab at click, been wanting to learn it anyways.

[VDFaller]

@z3z1ma I moved the doc string to raw, so I can make a test. But the below doesn't work. The click is going through so maybe I don't have my environment setup correctly?

cd demo_duckdb
dbt-osmosis yaml document orders --use-direct-yaml-descriptions

Am I missing something? As knowledge and whatnot looks right to me. But when I look at status in the schema file it just adds data type with no new description.

      - name: status
        tests:
          - accepted_values:
              values: ['placed', 'shipped', 'completed', 'return_pending', 'returned']
        data_type: string

[z3z1ma]

I can try it on my environment to see if I can sniff anything out. Looks right AFAICT.

[z3z1ma]

This section
https://github.com/VDFaller/dbt-osmosis/blob/inherit-from-yaml/src/dbt_osmosis/core/column_level_knowledge.py#L28-L32
Needs to be updated to include seeds
so k["progenitor"].startswith("source")
should be (k["progenitor"].startswith("source") or k["progenitor"].startswith("seed")) since a seed is effectively a source, meaning its always a base node

I added that and validated the the description was propagated successfully as a doc block @VDFaller

[z3z1ma]

Just a useful tidbit. If you are in the demo_duckdb directory, you can do dbt build -t test and it works out of the box. Just delete the existing .duckdb file if its there. After that you can dbt-osmosis yaml refactor/document/organize -t test [other args]

[z3z1ma]

LGTM, as I said in our slack chain -- fantastic work

[z3z1ma]

Ill handle documentation update and version bump for pypi

[VDFaller]

Sorry I never got back to you. yeah I'm all good.