Ask Your Question
1

read mysql bin log failed?

asked 2017-09-14 00:57:14 -0500

casel.chen gravatar image

updated 2019-06-21 12:15:16 -0500

metadaddy gravatar image

I want to setup a pipeline from mysql bin log origin to kafka producer destination. In latest streamset dc 2.7.1 web UI it show the error

"Pipeline Status: RUNNING_ERROR: com.streamsets.pipeline.api.StageException: MYSQL_003 - Error connecting to MySql: java.util.concurrent.TimeoutException: BinaryLogClient was unable to connect in 5000ms",

and I noticed the console log as following:

Exception in thread "blc-xxx.xxx.xxx.xxx:3306" java.lang.NumberFormatException: For input string: "
b"
 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:569)
 at com.github.shyiko.mysql.binlog.network.protocol.command.DumpBinaryLogGtidCommand.hexToByteArray(DumpBinaryLogGtidCommand.java:73)
 at com.github.shyiko.mysql.binlog.network.protocol.command.DumpBinaryLogGtidCommand.toByteArray(DumpBinaryLogGtidCommand.java:59)
 at com.github.shyiko.mysql.binlog.network.protocol.PacketChannel.write(PacketChannel.java:65)
 at com.github.shyiko.mysql.binlog.network.protocol.PacketChannel.write(PacketChannel.java:75)
 at com.github.shyiko.mysql.binlog.BinaryLogClient.requestBinaryLogStream(BinaryLogClient.java:486)
 at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:431)
 at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:623)
 at java.lang.Thread.run(Thread.java:745)

The MySQL db read is from Ali Cloud. I also tried the pipeline in my local MySQL db, it works fine. Btw, I can run Maxwell (http://maxwells-daemon.io/) to read bin log from Ali MySQL db.

I want to find the root cause and fix it, but I don't know how to debug streamsets DC in local. Anyone can help me? Thanks!

edit retag flag offensive close merge delete

Comments

Can you post a stack trace as well?

Jisun gravatar imageJisun ( 2017-09-18 13:01:01 -0500 )edit

2 Answers

Sort by » oldest newest most voted
1

answered 2017-09-18 12:34:20 -0500

junko_urata gravatar image

The NumberFormatException is thrown from the code where it tries to convert uuid to integer. For some reason your uuid contains a character. This is the exact code that throws the exception (byte)Integer.parseInt(uuid.charAt(j) + "" + uuid.charAt(j + 1), 16); I don't know why this happens. Seems like you received a reply for how to debug StreamSets in local environment. I recommend you to debug and compare the UUID when you connect to your local MySQL and the one on Ali Cloud.

edit flag offensive delete link more
0

answered 2019-06-20 05:17:35 -0500

your mysql's version is 5.6+? try to

server-id=1
log-bin=mysql-bin
binlog_format=ROW
sync_binlog=1
binlog_gtid_simple_recovery=ON
log_slave_updates=ON
gtid_mode=ON
enforce_gtid_consistency=ON

作者:赵安家
链接:https://juejin.im/post/5d0b5bbcf265da1b7f29850a
来源:掘金
著作权归作者所有。商业转载请联系作者获得授权,非商业转载请注明出处。

edit flag offensive delete link more

Comments

Thanks for answering the question! Could you please edit your answer and translate it to English? Thanks!

metadaddy gravatar imagemetadaddy ( 2019-06-21 12:14:51 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-09-14 00:57:14 -0500

Seen: 554 times

Last updated: Jun 21