Ask Your Question
1

read mysql bin log failed?

asked 2017-09-14 00:57:14 -0600

casel.chen gravatar image

I want to setup a pipeline from mysql bin log origin to kafka producer destination. In latest streamset dc 2.7.1 web UI it show the error "Pipeline Status: RUNNING_ERROR: com.streamsets.pipeline.api.StageException: MYSQL_003 - Error connecting to MySql: java.util.concurrent.TimeoutException: BinaryLogClient was unable to connect in 5000ms",

and I noticed the console log as following:

Exception in thread "blc-xxx.xxx.xxx.xxx:3306" java.lang.NumberFormatException: For input string: " b" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:569) at com.github.shyiko.mysql.binlog.network.protocol.command.DumpBinaryLogGtidCommand.hexToByteArray(DumpBinaryLogGtidCommand.java:73) at com.github.shyiko.mysql.binlog.network.protocol.command.DumpBinaryLogGtidCommand.toByteArray(DumpBinaryLogGtidCommand.java:59) at com.github.shyiko.mysql.binlog.network.protocol.PacketChannel.write(PacketChannel.java:65) at com.github.shyiko.mysql.binlog.network.protocol.PacketChannel.write(PacketChannel.java:75) at com.github.shyiko.mysql.binlog.BinaryLogClient.requestBinaryLogStream(BinaryLogClient.java:486) at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:431) at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:623) at java.lang.Thread.run(Thread.java:745)

The MySQL db read is from Ali Cloud. I also tried the pipeline in my local MySQL db, it works fine. Btw, I can run Maxwell (http://maxwells-daemon.io/) to read bin log from Ali MySQL db.

I want to find the root cause and fix it, but I don't know how to debug streamsets DC in local. Anyone can help me? Thanks!

edit retag flag offensive close merge delete

Comments

Can you post a stack trace as well?

Jisun gravatar imageJisun ( 2017-09-18 13:01:01 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2017-09-18 12:34:20 -0600

junko_urata gravatar image

The NumberFormatException is thrown from the code where it tries to convert uuid to integer. For some reason your uuid contains a character. This is the exact code that throws the exception (byte)Integer.parseInt(uuid.charAt(j) + "" + uuid.charAt(j + 1), 16); I don't know why this happens. Seems like you received a reply for how to debug StreamSets in local environment. I recommend you to debug and compare the UUID when you connect to your local MySQL and the one on Ali Cloud.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-09-14 00:57:14 -0600

Seen: 77 times

Last updated: Sep 18 '17