Ask Your Question

How do I perform a lookup against a CSV file?

asked 2017-11-15 20:24:39 -0600

shelv gravatar image

updated 2017-11-16 11:58:52 -0600

metadaddy gravatar image

I have a CSV file that I need to perform a lookup against (similar to a JDBC/HBase lookup) where the first column is the key. How can I do this in my pipeline?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2017-11-16 11:53:42 -0600

metadaddy gravatar image

updated 2017-11-16 12:26:13 -0600

You can't do this directly, but you have some options:

  • Use a Python, Groovy, or JavaScript Evaluator, or even a custom Java processor, to read it in, parse it, put it in a map, and do the lookup.
  • Create a pipeline to convert the CSV to JSON, then copy/paste the JSON into a Static Lookup processor.
  • Create a pipeline to load the CSV into a supported lookup store (JDBC / HBase / Redis etc) and do the lookup there

The first two are only viable if the file is small, i.e. its contents will fit into memory.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-11-15 20:24:39 -0600

Seen: 1,098 times

Last updated: Nov 16 '17