Ask Your Question
1

primary key columns are missing in records: []

asked 2020-06-15 16:24:16 -0500

Grimur gravatar image

updated 2020-06-15 16:25:03 -0500

I am running a Oracle CDC->Field renamer->JDBC Producer pipeline

It was running for 3 days with data then I get the error, "primary key columns are missing in records: []" and afterwards "Record cannot be null" which crashes the pipeline.

The JDBC Producer is in MySQL.

I have read that adding primary key to the destination table could fix the issue, but I don't understand how the source table that doesn't have a primary key would need the destination table to have a primary key, that doesn't make any logical sense ?

edit retag flag offensive close merge delete

Comments

I would recommend you add a primary key on both source and destination tables.

KateF gravatar imageKateF ( 2020-06-16 08:13:01 -0500 )edit

For example: ALTER TABLE [source table] ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;

KateF gravatar imageKateF ( 2020-06-16 16:51:39 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2020-06-16 16:50:57 -0500

KateF gravatar image

Hi Grimur, actually I just encountered this error on one of my Oracle CDC pipelines, and both the source and destination tables had pks already. Problem was an update had come through for a table for which I had not enabled supplemental logging. Solution was to enable table logging per docs.

edit flag offensive delete link more

Comments

Hi KateF, I checked the tables with select * from ALL_LOG_GROUPS where LOG_GROUP_TYPE='ALL COLUMN LOGGING' and they are correct. One thing I found out was oracle database is using version 10G which could cause problems.

Grimur gravatar imageGrimur ( 2020-06-22 04:24:20 -0500 )edit

I notice also today that when I went to main dashboard, it would say pipe has been running for 2 days. When I click on the pipeline dashboard to see further details, it would immediately throw failed record cannot be null. I have a feeling that preview mode automatically triggers bias records ?

Grimur gravatar imageGrimur ( 2020-06-22 04:29:15 -0500 )edit

Grimur, so are you saying your database is actually Oracle 10G?

KateF gravatar imageKateF ( 2020-06-22 09:26:13 -0500 )edit

KateF, The Oracle CDC is a Oracle 10G database and JDBC Producer is MySQL.

Grimur gravatar imageGrimur ( 2020-06-22 09:33:06 -0500 )edit
1

I also found out that Oracle CDC "unsupported field type" would throw an error from a column type"BLOB" This is true, but later in the pipeline I am ignoring the column. I ended up setting "unsupported field type" to "send record to pipeline" as a field remover takes care of it.

Grimur gravatar imageGrimur ( 2020-06-23 04:09:05 -0500 )edit
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2020-06-15 16:24:16 -0500

Seen: 117 times

Last updated: Jun 15