Ask Your Question
1

How to do external data source lookup cached by Redis?

asked 2018-03-28 08:59:02 -0500

casel.chen gravatar image

updated 2018-03-28 09:00:53 -0500

I want to enrich data with external data source lookup, and hope to add a redis cache between streaming data and http client, if hit the cache then no need do expensive http client lookup, if not then send http request and when get data back just put it into redis cache for next lookup, how to achieve that in SDC?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2018-03-29 22:40:24 -0500

metadaddy gravatar image

Use the Redis Lookup Processor to lookup the key, then use a Stream Selector to test for the existence of the value with ${record:exists('/fieldname')}. Send records with the value on to the next stage, while records without the value can pass through the HTTP Client Processor. Something like this:

image description

edit flag offensive delete link more

Comments

Thanks! You saved my day.

casel.chen gravatar imagecasel.chen ( 2018-03-29 23:55:37 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-03-28 08:59:02 -0500

Seen: 25 times

Last updated: Mar 29