JDBC driver in full mode keep on readign the data.

asked 2020-07-09 09:04:05 -0500

parakh gravatar image

Hi All,

i am running SDC 3.16.0 in a docker, the issue i am facing when i need to move a SQL query from a Parallel Data warehouse (PDW connected via mysql driver) to postgress db. I have set up jdbc query consumer with an event with pipeline finisher. The query should fetched some 260000 records. When running this pipeline i see it keeps on running infinitely and crosses some 11 HRS still running. and read more than 11 lakh records.

when i am running it with a smaller number of 500 or so, it work fine.

edit retag flag offensive close merge delete


Can you confirm that the data is not being actively inserted into the source db when you run the pipeline?

iamontheinet gravatar imageiamontheinet ( 2020-07-09 10:59:43 -0500 )edit