Ask Your Question
0

jdbc producer does not load data

asked 2020-11-05 23:45:06 -0600

Allen gravatar image

I am testing loading data from jdbc multitable consumer origin to jdbc producer destination. jdbc multitable source is mysql 5.7 and jdbc producer destination mariadb server 10.3. Validation was done correctly, and preview shows data from source, but data from source serer is not loaded to target server.

I don't know what I missed here. log shows "No More data to process, Triggered No More Data Event" Let me share my config of both source and origin along with logs.

config capture log: 2020-11-05 21:22:56,345 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO HikariPool-13 - Starting... HikariDataSource
2020-11-05 21:22:56,355 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO HikariPool-13 - Start completed. HikariDataSource
2020-11-05 21:22:56,388 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Selected Tables: AbstractTableJdbcSource sbtest1 _sbtest1_del 2020-11-05 21:22:56,388 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Ordering of Tables : TableOrderProvider
sbtest1 _sbtest1_del 2020-11-05 21:22:56,388 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Database Product name: MySQL JdbcUtil
2020-11-05 21:22:56,388 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Database product version: 5.7.24-log JdbcUtil
2020-11-05 21:22:56,388 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Driver name: MariaDB Connector/J JdbcUtil
2020-11-05 21:22:56,388 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Driver version: 2.5.4 JdbcUtil
2020-11-05 21:22:56,393 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Currently Registered JDBC drivers: HikariPoolConfigBean *xxxxxx 0 ProductionPipelineRunnable-schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d-schema_sync 2020-11-05 21:22:56,394 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Driver class org.mariadb.jdbc.Driver (version 2.5) from file:/opt/local/streamsets-datacollector-3.19.0/streamsets-libs/streamsets-datacollector-jdbc-lib/lib/mariadb-java-client-2.5.4.jar HikariPoolConfigBean *xxxxxx 0 ProductionPipelineRunnable-schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d-schema_sync 2020-11-05 21:22:56,394 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Driver class com.microsoft.sqlserver.jdbc.SQLServerDriver (version 7.2) from file:/opt/local/streamsets-datacollector-3.19.0/streamsets-libs/streamsets-datacollector-jdbc-lib/lib/mssql-jdbc-7.2.2.jre8.jar HikariPoolConfigBean *xxxxxx 0 ProductionPipelineRunnable-schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d-schema_sync 2020-11-05 21:22:56,394 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Driver class com.mysql.jdbc.Driver (version 5.1) from file:/opt/local/streamsets-datacollector-3.19.0/streamsets-libs/streamsets-datacollector-jdbc-lib/lib/mysql-connector-java-5.1.42-bin.jar HikariPoolConfigBean *xxxxxx 0 ProductionPipelineRunnable-schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d-schema_sync 2020-11-05 21:22:56,394 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Driver class com.mysql.fabric.jdbc.FabricMySQLDriver (version 5.1) from file:/opt/local/streamsets-datacollector-3.19.0/streamsets-libs/streamsets-datacollector-jdbc-lib/lib/mysql-connector-java-5.1.42-bin.jar HikariPoolConfigBean *xxxxxx 0 ProductionPipelineRunnable-schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d-schema_sync 2020-11-05 21:22:56,394 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO Driver class org.postgresql.Driver (version 42.2) from file:/opt/local/streamsets-datacollector-3.19.0/streamsets-libs/streamsets-datacollector-jdbc-lib/lib/postgresql-42.2.2.jar HikariPoolConfigBean *xxxxxx 0 ProductionPipelineRunnable-schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d-schema_sync 2020-11-05 21:22:56,394 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO HikariPool-14 - Starting... HikariDataSource *xxxxxx 0 ProductionPipelineRunnable-schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d-schema_sync 2020-11-05 21:22:56,404 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO HikariPool-14 - Start completed. HikariDataSource *xxxxxx 0 ProductionPipelineRunnable-schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d-schema_sync 2020-11-05 21:22:56,624 schema_sync/schemasyn27eb56de-57d6-43e8-b1b0-fea50ec8c20d INFO No More data to process, Triggered No More Data Event JdbcUtil

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2020-11-09 12:38:48 -0600

jeff gravatar image

From the description, I'm guessing the offset was already advanced to the end of the table from a previous run. You can reset the offset and try again.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-11-05 23:43:52 -0600

Seen: 183 times

Last updated: Nov 09