You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
DBT Core works properly with replicated tables in ClickHouse, and DBT Core does not randomly freeze while updating replicated tables.
Expected Behavior
DBT Core is unable to update tables with the Replicated engine, as DBT creates an empty copy of the destination table and, after completing the update (in my case, with materialized='incremental' and incremental_strategy='delete+insert', inserts data from the copy into the destination table and then deletes the copy. However, this process ignores the fact that Clickhouse is unable to create a copy of a replicated table, resulting in an exception being thrown (Code: 253. DB::Exception: Replica /clickhouse/tables/db/table/replicas/replica_name already exists).
Although this was expected, I attempted to use the MergeTree engine for updating, since it does not require replicating the table with new data. However, unexpectedly, DBT begins to freeze randomly while performing the updates. Sometimes, it works fine, but at other times, it appears to be deadlocked for some reason, regardless of the number of threads I use.
If this issue is not related to the DBT team, the problem with the 253 exception may not be specific to this repository. However, the freezing issue may be due to some other factor.
Is this a new bug in dbt-core?
Current Behavior
DBT Core works properly with replicated tables in ClickHouse, and DBT Core does not randomly freeze while updating replicated tables.
Expected Behavior
DBT Core is unable to update tables with the Replicated engine, as DBT creates an empty copy of the destination table and, after completing the update (in my case, with
materialized='incremental' and incremental_strategy='delete+insert'
, inserts data from the copy into the destination table and then deletes the copy. However, this process ignores the fact that Clickhouse is unable to create a copy of a replicated table, resulting in an exception being thrown (Code: 253. DB::Exception: Replica /clickhouse/tables/db/table/replicas/replica_name already exists
).Although this was expected, I attempted to use the MergeTree engine for updating, since it does not require replicating the table with new data. However, unexpectedly, DBT begins to freeze randomly while performing the updates. Sometimes, it works fine, but at other times, it appears to be deadlocked for some reason, regardless of the number of threads I use.
If this issue is not related to the DBT team, the problem with the 253 exception may not be specific to this repository. However, the freezing issue may be due to some other factor.
Steps To Reproduce
create Replicated table
run update with incremental strategy
create replicated table
switch model engine to not replicated
run update with incremental strategy
Relevant log output
No response
Environment
Which database adapter are you using with dbt?
other (mention it in "Additional Context")
Additional Context
dbt-clickhouse: 1.8.4
The text was updated successfully, but these errors were encountered: