Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

MongoDB Destination

Hi,

I am trying to load the data into MongoDB destination. From the data preview I am getting the data as below for MongoDB destination: Input Data
Record1 : {MAP} _id : {STRING} 5c0481fceef182fcebc823b0 id : {STRING} 1 name : {STRING} null dept : {STRING} 10 sdc.mongodb.operation : {STRING} 1

I had configured field sdc.mongodb.opearation as per the documentation https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Destinations/MongoDB.html#concept_eth_k5n_4v.

But when I ran the pipeline, getting the error as below: MONGODB_13 - Error serializing record 'com.streamsets.pipeline.api.base.OnRecordErrorException: MONGODB_15 - Operation type (insert, update or delete) is not specified in the header for record mongodb_oplog::0::41::0': com.streamsets.pipeline.api.base.OnRecordErrorException: MONGODB_15 - Operation type (insert, update or delete) is not specified in the header for record mongodb_oplog::0::41::0

I had set unique key field as id (field is from incoming data) and checked the upsert option.

Please suggest me how to configure MongoDB destination (having sdc.operation.type header attribute is correct).

MongoDB Destination

Hi,

I am trying to load the data into MongoDB destination. From the data preview I am getting the data as below for MongoDB destination: Input Data
Record1 : {MAP} _id : {STRING} 5c0481fceef182fcebc823b0 id : {STRING} 1 name : {STRING} null dept : {STRING} 10 sdc.mongodb.operation : {STRING} 1

I had configured field sdc.mongodb.opearation as per the documentation https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Destinations/MongoDB.html#concept_eth_k5n_4v.

But when I ran the pipeline, getting the error as below: MONGODB_13 - Error serializing record 'com.streamsets.pipeline.api.base.OnRecordErrorException: MONGODB_15 - Operation type (insert, update or delete) is not specified in the header for record mongodb_oplog::0::41::0': com.streamsets.pipeline.api.base.OnRecordErrorException: MONGODB_15 - Operation type (insert, update or delete) is not specified in the header for record mongodb_oplog::0::41::0

I had set unique key field as id (field is from incoming data) and checked the upsert option.

Please suggest me how to configure MongoDB destination (having sdc.operation.type header attribute is correct).