Ask Your Question
0

How to create Job run id

asked 2018-05-21 23:02:31 -0500

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

I want to create a job run id in StreamSets. How do I do this?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2018-05-21 23:28:01 -0500

metadaddy gravatar image

Assuming you want a unique id for each run of a pipeline, you can use any of the script evaluators for this. For example, in Groovy, you can create a UUID in the Init Script, which runs once, at pipeline startup:

state['job_run_id'] = UUID.randomUUID().toString()

Now you can set a field value on each record in the main script:

for (record in records) {
  try {
    // Set Job Run ID in field
    record.value['job_run_id'] = state['job_run_id']

    // Write a record to the processor output
    output.write(record)
  } catch (e) {
    // Write a record to the error pipeline
    log.error(e.toString(), e)
    error.write(record, e.toString())
  }
}
edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-05-21 23:02:31 -0500

Seen: 18 times

Last updated: May 22