Ask Your Question

# Revision history [back]

Yes, this is possible. Use an Expression Evaluator between each destination and the executor to set a record field or attribute. Here's an example:

Here, the raw data source has some delimited test data:

test
1
-1


The Stream Selector simply compares the test field to 0. If the value is greater than zero, then the record is written to a file in JSON format via the upper Local FS destination, otherwise it's sent to the lower one. The two Expression Evaluators set a field in the record: /flag; the upper one sets it to 1. the lower to 2. The Shell Executor copies the values of the flag field and event type into the script's environment:

and its script writes their values to a file:

echo $FLAG,$EVENT_TYPE >> /tmp/out/flags


Running the pipeline results in the expected output:

file1

{"test":"1"}


file2

{"test":"-1"}


flags

1, file-closed
2, file-closed


Yes, this is possible. Use an Expression Evaluator between each destination and the executor to set a record field or attribute. Here's an example:

Here, the raw data source has some delimited test data:

test
1
-1


The Stream Selector simply compares the test field to 0. If the value is greater than zero, then the record is written to a file in JSON format via the upper Local FS destination, otherwise it's sent to the lower one. one.

The two Expression Evaluators set a field in the record: /flag; the upper one sets it to 1. the lower to 2. They also set a /timestamp field to ${time:now()}. The Shell Executor copies the values of the flag field and timestamp fields and event type into the script's environment: and its script writes their values to a file: echo$FLAG, $EVENT_TYPE$EVENT_TYPE, \$TIME_NOW >> /tmp/out/flags


Running the pipeline results in the expected output:

file1

{"test":"1"}


file2

{"test":"-1"}


flags

1, file-closed
file-closed, Wed Jan 17 08:16:01 PST 2018
2, file-closed

file-closed, Wed Jan 17 08:16:01 PST 2018