Ask Your Question
3

Unable to achieve CDC using MySQL Binary Log for Mariadb database.

asked 2018-01-03 01:21:52 -0500

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

I am trying to use MySQL Binary Log Origin for CDC ,which worked well for mysql.

Does the same origin supports mariadb as well? If not, which of the origin need to be used for CDC with mariadb. Please, someone suggest me solution for this.

Thanks

edit retag flag offensive close merge delete

Comments

Have you tried it? If so, what error did you see?

metadaddy gravatar imagemetadaddy ( 2018-01-12 13:09:29 -0500 )edit

yes I have tried,my pipeline is running without any errors.But no data is fetched. I tried updating,inserting,deleting data from table what i used as origin while i ran the pipeline.But streamsets pipeline does not show any data .

Maithri gravatar imageMaithri ( 2018-01-13 07:47:24 -0500 )edit

I created new pipeline with the same configurations as the previous pipeline,this time it gives error as unable to find driver instance !! But all the connectors are placed .

Maithri gravatar imageMaithri ( 2018-01-17 04:17:48 -0500 )edit

Thank you for your response @metadaddy

Maithri gravatar imageMaithri ( 2018-01-17 07:31:02 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2018-01-17 07:34:55 -0500

Maithri gravatar image

Though i was getting error as unable to find the driver,the issue was with the configuration file of mariadb. soln:

      /etc/mysql/my.cnf in local /etc/mysql/mariadb.cnf in server



    server-id               = 1
    log_bin                 = /var/log/mysql/mariadb-bin
    log_bin_index           = /var/log/mysql/mariadb-bin.index
    sync_binlog             = 1
    binlog_format=row
    innodb_flush_log_at_trx_commit=0

making these configuration solved my issue!

edit flag offensive delete link more

Comments

This pipeline captured changes made by update,insert and delete. But why is that the pipeline with mysqlbin origin is fetching only new data? How to fetch entire data with changed data as well?

Maithri gravatar imageMaithri ( 2018-01-17 07:37:24 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-01-03 01:21:52 -0500

Seen: 406 times

Last updated: Jan 22