Ask Your Question
1

Access to batch number and runner id

asked 2020-09-14 00:20:13 -0500

graphadvantage gravatar image

Hi

Does anybody know how to access the batch number for a batch (eg reading a csv in batch of 10000) and also how to access the thread/runner number that a batch is being processed in?

I'd like to fork data streams using a stream selector condition based on either batch number or runner number accessed using the expression language...

thanks

Michael

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2020-09-14 13:50:59 -0500

iamontheinet gravatar image

Hi Michael,

I don't believe those attributes are exposed/available to use. What you could do is keep a running counter in a state object in one of the scripting processors. Note that the state object is initialized when the pipeline starts so you can:

  • Set its default value to 1 in Init Script
  • Increment it in Script section and include it in the record header
  • Use the record header attribute to branch off data streams

Hope this helps.

Cheers, Dash

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-09-14 00:20:13 -0500

Seen: 22 times

Last updated: Sep 14