Ask Your Question
1

Writing nested json to Influx db destination

asked 2019-01-22 16:31:30 -0600

srinath_222 gravatar image

Hi,

I have a simple pipeline which reads from Kafka topic and loading to Influx DB destination.

I wanted to load the nested data into a single field in Influx DB destination. I am selecting the fields using Value Fields option in Influx DB destination. I have my nested data under Data field as shown in preview below: image description

I want the all the fields under /data to be in a single field in Influx DB destination, but getting the error as below: java.lang.RuntimeException: {"error":"unable to parse 'Address_cdc data={field1=Field[STRING:xyz], field2=Field[STRING:xyz2], field3=Field[STRING:xyz3]} 1548196236580000000': invalid boolean"}

Please suggest how to use Influx DB destination for these kind of scenarios.

edit retag flag offensive close merge delete

Comments

How do you want the three fields combined into one?

metadaddy gravatar imagemetadaddy ( 2019-01-22 16:40:12 -0600 )edit

I want the data in a single field in Influx DB like {"field1":"xyz","field2":"xyz2","field3":"xyz3"}

srinath_222 gravatar imagesrinath_222 ( 2019-01-22 16:49:35 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2019-01-22 19:07:22 -0600

metadaddy gravatar image

Since the Data Generator processor operates on the whole record, you'll need to use one of the script evaluators for this. Here is how to replace the /data field with its JSON encoding in the Groovy Evaluator:

import groovy.json.JsonOutput;

for (record in records) {
  try {
    // Replace the /data field with its JSON encoding
    record.value['data'] = JsonOutput.toJson(record.value['data'])

    // Write a record to the processor output
    output.write(record)
  } catch (e) {
    // Write a record to the error pipeline
    log.error(e.toString(), e)
    error.write(record, e.toString())
  }
}

In preview:

image description

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-01-22 16:31:30 -0600

Seen: 27 times

Last updated: Jan 22