sdc mysql binlog pipeline still running in background even after stopped

asked 2018-11-02 06:04:48 -0600

zhubin gravatar image

updated 2018-11-04 19:01:18 -0600

Hi, I have some problem using binlog orgin, the situation is I cannot connect to my mysql, ok , that`s not the key point, the key point is: when I stop this pipeline, it is still trying to connect to mysql in backgroud, I have to restart my sdc to resolve this.

by the way, my SDC version is 3.5.1, deployed by cloudera manager 5.15.1

Hope to find the reason, big thanks!

here is sdc.log:

## pipeline encounter problem connecting to mysql:

Caused by: com.github.shyiko.mysql.binlog.network.ServerException: A slave with the same server_uuid/server_id as this slave has connected to the master; the first event 'mysql_binary_log.000014' at 10308, the last event read from '/var/lib/mysql/mysql_binary_log.000014' at 123, the last byte read from '/var/lib/mysql/mysql_binary_log.000014' at 10308. at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:882) at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559) at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793) ... 1 more

##I stop the pipelint

2018-11-02 17:58:47,537 [user:admin] [pipeline:employee-binlog-01/employeebinlog016f647822-629f-4909-b762-7216a5e87308] [runner:] [thread:webserver-305] INFO StandaloneRunner - Preparing to stop pipeline

2018-11-02 17:58:47,573 [user:*admin] [pipeline:employee-binlog-01/employeebinlog016f647822-629f-4909-b762-7216a5e87308] [runner:] [thread:runnerStop-pool-10-thread-4] INFO StandaloneRunner - Stopping pipeline employeebinlog016f647822-629f-4909-b762-7216a5e87308 0

2018-11-02 17:58:47,573 [user:*admin] [pipeline:employee-binlog-01/employeebinlog016f647822-629f-4909-b762-7216a5e87308] [runner:] [thread:runnerStop-pool-10-thread-4] INFO ProductionPipelineRunnable - Pipeline is in terminal state

##pipeline still try to connect to mysql in background

2018-11-02 17:59:16,109 [user:*admin] [pipeline:employee-binlog-01/employeebinlog016f647822-629f-4909-b762-7216a5e87308] [runner:] [thread:blc-keepalive-a01.data-test.lgy.etcp.cn:3306] INFO BinaryLogClient - Trying to restore lost connection to a01.data-test.lgy.etcp.cn:3306

2018-11-02 17:59:16,171 [user:*admin] [pipeline:employee-binlog-01/employeebinlog016f647822-629f-4909-b762-7216a5e87308] [runner:] [thread:blc-a01.data-test.lgy.etcp.cn:3306] INFO BinaryLogClient - Connected to a01.data-test.lgy.etcp.cn:3306 at mysql_binary_log.000014/10308 (sid:1, cid:94231)

2018-11-02 17:59:36,754 [user:*admin] [pipeline:employee-binlog-01/employeebinlog016f647822-629f-4909-b762-7216a5e87308] [runner:] [thread:blc-keepalive-a01.data-test.lgy.etcp.cn:3306] INFO BinaryLogClient - Trying to restore lost connection to a01.data-test.lgy.etcp.cn:3306

2018-11-02 17:59:36,803 [user:*admin] [pipeline:employee-binlog-01/employeebinlog016f647822-629f-4909-b762-7216a5e87308] [runner:] [thread:blc-a01.data-test.lgy.etcp.cn:3306] INFO BinaryLogClient - Connected to a01.data-test.lgy.etcp.cn:3306 at mysql_binary_log.000014/10308 (sid:1, cid:94242)

2018-11-02 18:00:16,172 [user:*admin] [pipeline:employee-binlog-01/employeebinlog016f647822-629f-4909-b762-7216a5e87308] [runner:] [thread:blc-keepalive-a01.data-test.lgy.etcp.cn:3306] INFO BinaryLogClient - Trying to restore lost connection to a01.data-test.lgy.etcp.cn:3306

2018-11-02 18:00:16,233 [user:*admin] [pipeline:employee-binlog-01/employeebinlog016f647822-629f-4909-b762-7216a5e87308] [runner:] [thread:blc-a01.data-test.lgy.etcp

edit retag flag offensive close merge delete

Comments

Please indicate the SDC version

jeff gravatar imagejeff ( 2018-11-02 14:54:57 -0600 )edit

sorry, fogot to indicate version, thanks! my SDC version is 3.5.1, deployed by cloudera manager 5.15.1

zhubin gravatar imagezhubin ( 2018-11-04 19:00:30 -0600 )edit