mysql binlog gtid

asked 2019-07-09 02:50:02 -0500

lichanghong gravatar image

updated 2019-07-09 02:52:12 -0500

Hello,

I use the mysql binlog gitd as the origin. during the processing ,I found two issue.

  1. the offset.json file read like this : { "offsets" : { "$com.streamsets.datacollector.pollsource.offset$" : "{\"gtidSet\":\"0d3a5ec2-fa0d-11e7-821e-525400c799f2:1-12723000\",\"incompleteTransactions\":{\"0d3a5ec2-fa0d-11e7-821e-525400c799f2:9004528\":1,\"0d3a5ec2-fa0d-11e7-821e-525400c799f2:11496538\":3}}" }, "version" : 2 }

2. the log warn that The maximum number of records per batch in the origin has been exceeded.

my problem is that

1.in the offset.json file ,there are more than one incompleteTransactions position。 Does this lead to the data loss?

2. despite the log is level warn ,not error, I am still a little concerned whether this issue will lead to the data loss or not?

edit retag flag offensive close merge delete