Ask Your Question
0

Is it possible to have two sources ?

asked 2018-01-22 23:47:34 -0600

Shruthi gravatar image

updated 2018-01-25 07:35:50 -0600

I want to use 'JDBC Multitable' first to fetch full data from a table and later a 'JDBC Query' to filter data from this fetched one. That is, I just want to fetch and keep the entire data of a table first and then query on them later. How can i do this? I did not find any processor for querying so that i can avoid 'JDBC Query' and use a single source 'JDBC Multitable' .

I don't think it is possible to use two sources in a pipeline. If not what is the solution for my requirement? Or is it possible to connect two pipelines and run together

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2018-01-23 12:17:45 -0600

jeff gravatar image

See this question for some more background on why StreamSets Data Collector only supports a single origin per pipeline.

Given what you've outlined, is it possible to use the JDBC Lookup processor? That would allow you to execute a query to get additional data for each row in the data stream (i.e. your multi table origin).

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-01-22 23:47:34 -0600

Seen: 89 times

Last updated: Jan 25