Ask Your Question
1

How do I perform a lookup against a CSV file?

asked 2017-11-15 20:24:39 -0500

shelv gravatar image

updated 2017-11-16 11:58:52 -0500

metadaddy gravatar image

I have a CSV file that I need to perform a lookup against (similar to a JDBC/HBase lookup) where the first column is the key. How can I do this in my pipeline?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2017-11-16 11:53:42 -0500

metadaddy gravatar image

updated 2017-11-16 12:26:13 -0500

You can't do this directly, but you have some options:

  • Use a Python, Groovy, or JavaScript Evaluator, or even a custom Java processor, to read it in, parse it, put it in a map, and do the lookup.
  • Create a pipeline to convert the CSV to JSON, then copy/paste the JSON into a Static Lookup processor.
  • Create a pipeline to load the CSV into a supported lookup store (JDBC / HBase / Redis etc) and do the lookup there

The first two are only viable if the file is small, i.e. its contents will fit into memory.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-11-15 20:24:39 -0500

Seen: 342 times

Last updated: Nov 16 '17