Fix Python regressions in 1.9.0beta #857
Conversation
benc-db
requested review from
andrefurlan-db and
rcypher-databricks
as code owners
| if self.status(cluster_id) not in ["RUNNING", "PENDING"]: | ||
| raise DbtRuntimeError(f"Error starting terminated cluster.\n {response.content!r}") | ||
| else: | ||
| logger.debug("Presuming race condition, waiting for cluster to start") |
There was a problem hiding this comment.
When we query state after error, if it's pending or running it means the start failed due to race condition as another thread got the cluster started.
There was a problem hiding this comment.
should it be info rather than debug or debug with status_code?
There was a problem hiding this comment.
Anything that is info will show in the normal dbt output, so we are very conservative about what we log with info.
This is normal operation; if they are using multiple python models and the command api they will hit this, so I don't think this is worth bringing to the users' attention if it's running or pending. For whatever reason, the cluster start API errors if you ask to start a cluster that is already in the process of starting. If it's not running or pending, they'll get the full output from raising the error.
| ) | ||
| ).json() | ||
|
|
||
| if response["results"]["resultType"] == "error": |
There was a problem hiding this comment.
Lost this in the refactor: for some reason Command exec will give a state of 'Finished' rather then 'Error' some times, and then stuff the error in the results.
There was a problem hiding this comment.
what if the result or resultType does not exist
There was a problem hiding this comment.
these are published parts of the Databricks REST API. If these don't exist, the entire API becomes untrustworthy.
| @@ -85,7 +85,7 @@ | |||
| SHOW_TABLE_EXTENDED_MACRO_NAME = "show_table_extended" | |||
| SHOW_TABLES_MACRO_NAME = "show_tables" | |||
| SHOW_VIEWS_MACRO_NAME = "show_views" | |||
| GET_COLUMNS_COMMENTS_MACRO_NAME = "get_columns_comments" | |||
|
|
|||
There was a problem hiding this comment.
Noticed this wasn't referenced anywhere when doing my debugging.
| @@ -70,6 +71,8 @@ def __init__( | |||
|
|
|||
| @override | |||
| def submit(self, compiled_code: str) -> None: | |||
| logger.debug("Submitting Python model using the Command API.") | |||
There was a problem hiding this comment.
Adding debug statement so that we can quickly determine submission method from logs.
| @@ -0,0 +1,27 @@ | |||
|
|
|||
There was a problem hiding this comment.
Moved these to somewhere more sensible (during investigation realized these were in persist_docs, but are sometimes called outside of persisting docs).
Description
Fixing two regressions related to Command submission of python models.
1.) Exceptions were not properly getting propagated to the user
2.) I had accidentally brought back the race condition that the community had already fixed for us.
Checklist
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section.