Error with Google BigQuery destination
I am using Kafka Consumer as the origin and Google Bigquery as the destination (with StreamSets DataCollector 2.7.2.0) of my pipeline. The Data Format at the origin is JSON. I get the following error:
'Root field of record should be a Map or a List Map'
How can I resolve this issue?
What is the type of record you are trying to write? If you do a preview, you should easily be able to see. My guess is that you are consuming from Kafka as TEXT (the default) and hence the root field is simply a STRING.
Thanks for your suggestion. The data format at Kafka Consumer stage is already JSON. Setting the JSON Content option to 'JSON array of objects' instead of 'Multiple JSON objects' solved the issue.