Ask Your Question
2

Why doesn't MySQL Binary Log fetch existing data?

asked 2018-01-17 07:28:32 -0600

Shruthi gravatar image

updated 2018-02-05 07:16:56 -0600

I am using MySQL Binary Log to implement CDC. When i run the pipeline for the first time i get no records. But when i make some changes in the table, i.e if i perform some operations on my table that is reflecting in the output but not the entire data

For eg:- suppose i have 25 records in my table. I want to fetch all 25 records first and later the changes i make to any of the records. But i am not able to fetch all 25 records first.Only the changes i am able to capture.

Why is it happening so? Can anyone suggest some way to solve this?

edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted
0

answered 2018-01-17 12:08:06 -0600

metadaddy gravatar image

The purpose of the CDC origins (MySQL Binary Log, Oracle CDC Client etc) is to ingest changes as they happen. You should use a separate pipeline with the JDBC Multitable Consumer to fetch existing data.

edit flag offensive delete link more

Comments

But sometimes in the first run it fetches the exiting data along with the changed data as new records. And if i want to use a separate pipeline how can i do that?

Shruthi gravatar imageShruthi ( 2018-01-17 23:18:15 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-01-17 07:28:32 -0600

Seen: 58 times

Last updated: Feb 05