Ask Your Question

How to filter records received by HTTP Server origin

asked 2018-03-02 00:33:48 -0600

lalatdas gravatar image

updated 2018-03-02 10:11:36 -0600

metadaddy gravatar image

While experimenting with StreamSets mentioned the origin as http server which is supposed to push line protocols to influxdb after doing filtration. The pipeline looks something like below.

httpserver ---> **Filter only record where a particular field's value is less than a number** ---> httpClient---> influxDB

sales  tv=23, paper=34, radio=54  sales_val=17
sales  tv=23, paper=34, radio=54  sales_val=13

If we send the above line to httpserver from an outside http client, and we filter only records whose sales_val are more than 14. I am unable to find any filter_record type of processor, though we have processor excellent spectrum of field modifying processors. Please let me know if I understood anything wrt record and field.

Please let me know if we have any other way (new type of pipeline) to achieve the same.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-03-02 10:10:41 -0600

metadaddy gravatar image

You can use the Stream Selector processor to do this - set the condition for the first stream to something like:

${record:value('/sales_val') > 14}

You can then send stream 2 straight to 'Trash' or process it some other way.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools



Asked: 2018-03-02 00:33:48 -0600

Seen: 212 times

Last updated: Mar 02 '18