Ask Your Question

How can I differentiate the data coming from multiple HTTP client origins

asked 2017-12-07 08:09:02 -0500

Roh gravatar image

updated 2017-12-07 11:47:48 -0500

metadaddy gravatar image

I have 6 pipelines each have the HTTP client origin connected to the SDCRPC destinations, my plan is to make another pipeline with SDCRPC origin and destination to hive tables.

My question is after connecting to the SDCRPC origin how can I differentiate the data because each HTTP pipeline pulls the data related to one identical table.

Any examples or online resources will be appreciated.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2017-12-07 11:50:16 -0500

metadaddy gravatar image

Add an expression evaluator to each of the 6 HTTP->SDCRPC pipelines, each of them adding a different header attribute value to the records. Now your SDCRPC origin pipeline can look at that header attribute to figure out which pipeline it came from.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-12-07 08:09:02 -0500

Seen: 1,075 times

Last updated: Dec 07 '17