apache beam integration with streamset

asked 2018-07-19 09:01:49 -0500

ravi gravatar image

Hi, To avoid writing the code on multiple frameworks(spark,flink,..) and batch/streaming we are thinking of using Apache beam SDK. During this study, I came across the streamset SDK. Can you share the commonality, differences between these two and how it can be integrated together?. I meant can i use the beam sdk workflow as streamset processor and run it on spark/flink using wrapper streamset pipleline? E.g Streeam set pipe line -> bean pipeline processor -> flink runner -> outputs to kafka topic/elastic search?

Regards Ravi.P

edit retag flag offensive close merge delete