Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clickhouse/sink: alter add unknow default for column failed to applied #90

Open
laskoviymishka opened this issue Nov 6, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@laskoviymishka
Copy link
Contributor

laskoviymishka commented Nov 6, 2024

When postgres source got more then 1 columns to add replication generate an error:

error failed to execute pg activate hook: 
Snapshot loading failed: unable to upload tables: 
upload of 9 tables failed: 
unable to finish table loading: 
unable to sent 'done_load_table' for table '"public"."companies" [1/1] (ETARows: 8)': unable push batch with a non row item: 
failed to push items from 0 to 8 in batch: 
failed to push (with retries): Push failed: 
failed to push 8 rows to ClickHouse shard 0: 
ClickHouse Push failed (got fatal error): unable to get table 'public_companies': 
failed to apply schema diff to public_companies: 
error executing DDL: 
code: 62, message: Syntax error: failed at position 200 (']'): ], 'Nullable(String)'), ADD COLUMN IF NOT EXISTS `default_data_language` Nullable(String), ADD COLUMN IF NOT EXISTS `additional_data_languages` Nullable(String). Expected one of: token, Comma

Here is a DDL:

ALTER TABLE `public_companies`  ON CLUSTER `default` 
ADD COLUMN IF NOT EXISTS `default_language` Nullable(String), 
ADD COLUMN IF NOT EXISTS `additional_languages` Nullable(String) 
     DEFAULT CAST(ARRAY[], 'Nullable(String)'), 
ADD COLUMN IF NOT EXISTS `default_data_language` Nullable(String), 
ADD COLUMN IF NOT EXISTS `additional_data_languages` Nullable(String) 
     DEFAULT CAST(ARRAY[], 'Nullable(String)')
@laskoviymishka laskoviymishka added the bug Something isn't working label Nov 6, 2024
@laskoviymishka laskoviymishka changed the title Alter add multiple columns failed to applied to clickhouse Alter add unknow default for column failed to applied to clickhouse Nov 6, 2024
@laskoviymishka
Copy link
Contributor Author

Postgres source schema:

Column           |            Type             | Collation | Nullable |                  Default                  
---------------------------+-----------------------------+-----------+----------+-------------------------------------------
 company_id                | text                        |           | not null | 
 external_id               | text                        |           | not null | 
 title                     | text                        | en_US     | not null | 
 fullname                  | text                        | en_US     |          | 
 user_id                   | text                        |           |          | 
 updated                   | timestamp(0) with time zone |           | not null | now()
 created                   | timestamp(0) with time zone |           | not null | now()
 lsn                       | bigint                      |           | not null | nextval('companies_lsn_seq'::regclass)
 serial                    | bigint                      |           | not null | nextval('companies_serial_seq'::regclass)
 secret                    | text                        |           |          | 
 products_scope            | text[]                      |           | not null | ARRAY[]::text[]
 ownership                 | text                        |           | not null | 'yandex'::text
 instance_erp              | text                        |           |          | 
 users_manage              | text                        |           |          | 
 vars                      | jsonb                       |           | not null | '{}'::jsonb
 source                    | text                        |           |          | 
 country_code              | text                        |           | not null | 
 type                      | text                        |           | not null | 'original'::text
 default_language          | text                        |           |          | 
 additional_languages      | text[]                      |           |          | ARRAY[]::text[]
 default_data_language     | text                        |           |          | 
 additional_data_languages | text[]                      |           |          | ARRAY[]::text[]

@laskoviymishka laskoviymishka changed the title Alter add unknow default for column failed to applied to clickhouse Clickhouse/sink: alter add unknow default for column failed to applied Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant