Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] clickhouse replicated error & freeze #11017

Closed
2 tasks done
V4kodin opened this issue Nov 20, 2024 · 1 comment
Closed
2 tasks done

[Bug] clickhouse replicated error & freeze #11017

V4kodin opened this issue Nov 20, 2024 · 1 comment
Labels
bug Something isn't working wontfix Not a bug or out of scope for dbt-core

Comments

@V4kodin
Copy link

V4kodin commented Nov 20, 2024

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

DBT Core works properly with replicated tables in ClickHouse, and DBT Core does not randomly freeze while updating replicated tables.

Expected Behavior

DBT Core is unable to update tables with the Replicated engine, as DBT creates an empty copy of the destination table and, after completing the update (in my case, with materialized='incremental' and incremental_strategy='delete+insert', inserts data from the copy into the destination table and then deletes the copy. However, this process ignores the fact that Clickhouse is unable to create a copy of a replicated table, resulting in an exception being thrown (Code: 253. DB::Exception: Replica /clickhouse/tables/db/table/replicas/replica_name already exists).

Although this was expected, I attempted to use the MergeTree engine for updating, since it does not require replicating the table with new data. However, unexpectedly, DBT begins to freeze randomly while performing the updates. Sometimes, it works fine, but at other times, it appears to be deadlocked for some reason, regardless of the number of threads I use.

If this issue is not related to the DBT team, the problem with the 253 exception may not be specific to this repository. However, the freezing issue may be due to some other factor.

Steps To Reproduce

  1. create Replicated table

  2. run update with incremental strategy

  3. create replicated table

  4. switch model engine to not replicated

  5. run update with incremental strategy

Relevant log output

No response

Environment

- OS: Ubuntu 22.04.5 LTS
- Python: 3.11.10
- dbt: 1.8.8
- clickhouse: 24.8.2.3

Which database adapter are you using with dbt?

other (mention it in "Additional Context")

Additional Context

dbt-clickhouse: 1.8.4

@V4kodin V4kodin added bug Something isn't working triage labels Nov 20, 2024
@V4kodin V4kodin changed the title [Bug] <clickhouse replicated error & freeze> [Bug] clickhouse replicated error & freeze Nov 20, 2024
@dbeatty10
Copy link
Contributor

Thanks for reaching out @V4kodin !

This looks specific to the dbt-clickhouse adapter, so could you please open an issue here instead?

Closing this report in the dbt-core repository in favor of creating one in dbt-clickhouse instead.

@dbeatty10 dbeatty10 closed this as not planned Won't fix, can't repro, duplicate, stale Nov 21, 2024
@dbeatty10 dbeatty10 added wontfix Not a bug or out of scope for dbt-core and removed triage labels Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working wontfix Not a bug or out of scope for dbt-core
Projects
None yet
Development

No branches or pull requests

2 participants