Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error parsing character varying data (postgres) #960

Open
subkanthi opened this issue Dec 18, 2024 · 1 comment
Open

Error parsing character varying data (postgres) #960

subkanthi opened this issue Dec 18, 2024 · 1 comment
Labels
lightweight Issues related to Lightweight version not reproducible Cannot reproduce the bug. postgres Issues related to PostgreSQL
Milestone

Comments

@subkanthi
Copy link
Collaborator

character varying(255) we have data like "Payment of 888884444524 (BK:BK_MTN_MOMO_PULL)" . So if there are brackets in the data it is throwing below error:
2024-12-17 17:03:37.191 INFO - ****** Thread: Sink Connector thread-pool-1 Batch Size: 150000 ******
2024-12-17 17:03:37.227 ERROR - ClickHouseBatchRunnable exception - Task(0)
java.util.UnknownFormatConversionException: Conversion = ')'
at java.util.Formatter.parse(Formatter.java:2750) ~[?:?]
at java.util.Formatter.format(Formatter.java:2671) ~[?:?]
at java.util.Formatter.format(Formatter.java:2625) ~[?:?]
at java.lang.String.format(String.java:4140) ~[?:?]
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.processRecordsByTopic(ClickHouseBatchRunnable.java:307) ~[app.jar:?]
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.run(ClickHouseBatchRunnable.java:188) [app.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
2024-12-17 17:03:50.169 INFO - ****** Thread: Sink Connector thread-pool-1 Batch Size: 150000 ******

@subkanthi subkanthi added lightweight Issues related to Lightweight version postgres Issues related to PostgreSQL labels Dec 18, 2024
@subkanthi subkanthi added this to the 2.6.0 milestone Dec 18, 2024
@subkanthi
Copy link
Collaborator Author

subkanthi commented Dec 18, 2024

public=# \d+ public.tm;
                                                                  Table "public.tm"
         Column          |           Type           | Collation | Nullable |      Default      | Storage  | Compression | Stats target | Description 
-------------------------+--------------------------+-----------+----------+-------------------+----------+-------------+--------------+-------------
 id                      | uuid                     |           | not null | gen_random_uuid() | plain    |             |              | 
 secid                   | uuid                     |           |          |                   | plain    |             |              | 
 acc_id                  | uuid                     |           |          |                   | plain    |             |              | 
 ccatz                   | character varying        |           |          |                   | extended |             |              | 
 tcred                   | boolean                  |           |          | false             | plain    |             |              | 
 am                      | numeric(21,5)            |           |          |                   | main     |             |              | 
 set_date                | timestamp with time zone |           |          |                   | plain    |             |              | 
 created                 | timestamp with time zone |           |          |                   | plain    |             |              | 
 updated                 | timestamp with time zone |           |          |                   | plain    |             |              | 
 events_id               | uuid                     |           |          |                   | plain    |             |              | 
 events_transaction_id   | uuid                     |           |          |                   | plain    |             |              | 
 events_status           | character varying        |           |          |                   | extended |             |              | 
public=# select * from public.tm;
                  id                  |                secid                 |                acc_id                |                 ccatz                  | tcred |      am      |           set_date           |           created            |           updated            |              events_id               |        events_transaction_id         | events_status | events_payment_snapshot |        events_created        |                 vid                  |                 vtid                 |  vstatus  |   vamount    |           vcreated           | vbilling_currency 
--------------------------------------+--------------------------------------+--------------------------------------+----------------------------------------+-------+--------------+------------------------------+------------------------------+------------------------------+--------------------------------------+--------------------------------------+---------------+-------------------------+------------------------------+--------------------------------------+--------------------------------------+-----------+--------------+------------------------------+-------------------
 9cb52b2a-8ef2-4987-8856-c79a1b2c2f71 | 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 | 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 | IDR 888884444524 (BK:BK_MTN_MOMO_PULL) | t     | 200000.00000 | 2022-10-16 20:53:15.01957+00 | 2022-10-16 20:53:15.01957+00 | 2022-10-16 20:53:15.01957+00 | b4763f4a-2e3f-41ae-9715-4ab113e2f53c | 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 |               | {"Hello": "World"}      | 2022-10-16 20:53:15.01957+00 |                                      |                                      |           |              |                              | 
 9cb52b2a-8ef2-4987-8856-c79a1b2c2f73 | 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 | 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 | IDR                                    | t     | 200000.00000 | 2022-10-16 20:53:15.01957+00 | 2022-10-16 20:53:15.01957+00 | 2022-10-16 20:53:15.01957+00 | b4763f4a-2e3f-41ae-9715-4ab113e2f53c | 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 |               |                         | 2022-10-16 20:53:15.01957+00 | 9cb52b2a-8ef2-4987-8856-c79a1b2c2f71 | 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 | COMPLETED | 200000.00000 | 2022-10-16 20:53:15.01957+00 | IDR
(2 rows)

select * from tm final;

SELECT *
FROM tm
FINAL

Query id: 420b984f-15df-4e1d-9eb3-86688b4f46b5

   ┌─id───────────────────────────────────┬─secid────────────────────────────────┬─acc_id───────────────────────────────┬─ccatz──────────────────────────────────┬─tcred─┬─────am─┬───────────────────set_date─┬────────────────────created─┬────────────────────updated─┬─events_id────────────────────────────┬─events_transaction_id────────────────┬─events_status─┬─events_payment_snapshot─┬─────────────events_created─┬─vid──────────────────────────────────┬─vtid─────────────────────────────────┬─vstatus───┬─vamount─┬───────────────────vcreated─┬─vbilling_currency─┬────────────_version─┬─_sign─┐
1. │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f71 │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 │ **IDR 888884444524 (BK:BK_MTN_MOMO_PULL)** │ true  │ 200000 │ 2022-10-16 20:53:15.019570 │ 2022-10-16 20:53:15.019570 │ 2022-10-16 20:53:15.019570 │ b4763f4a-2e3f-41ae-9715-4ab113e2f53c │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 │ ᴺᵁᴸᴸ          │ {"Hello": "World"}      │ 2022-10-16 20:53:15.019570 │ ᴺᵁᴸᴸ                                 │ ᴺᵁᴸᴸ                                 │ ᴺᵁᴸᴸ      │    ᴺᵁᴸᴸ │                       ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ              │ 1734558360304000001 │     0 │
2. │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f73 │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 │ IDR                                    │ true  │ 200000 │ 2022-10-16 20:53:15.019570 │ 2022-10-16 20:53:15.019570 │ 2022-10-16 20:53:15.019570 │ b4763f4a-2e3f-41ae-9715-4ab113e2f53c │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 │ ᴺᵁᴸᴸ          │ ᴺᵁᴸᴸ                    │ 2022-10-16 20:53:15.019570 │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f71 │ 9cb52b2a-8ef2-4987-8856-c79a1b2c2f72 │ COMPLETED │  200000 │ 2022-10-16 20:53:15.019570 │ IDR               │ 1734558360309000002 │     0 │
   └──────────────────────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┴────────────────────────────────────────┴───────┴────────┴────────────────────────────┴────────────────────────────┴────────────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┴───────────────┴─────────────────────────┴────────────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┴───────────┴─────────┴────────────────────────────┴───────────────────┴─────────────────────┴───────┘

@subkanthi subkanthi added the not reproducible Cannot reproduce the bug. label Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lightweight Issues related to Lightweight version not reproducible Cannot reproduce the bug. postgres Issues related to PostgreSQL
Projects
None yet
Development

No branches or pull requests

1 participant