Ask Your Question

Apache Flink/Kafka/Spark Streaming support in StreamSets

asked 2018-07-19 09:25:40 -0500

ravi gravatar image

updated 2018-07-20 15:18:00 -0500

metadaddy gravatar image

I have seen some documentation that we can use spark code inside stream set processor. But i am not clear whether we can use Flink/Kafka Streaming inside the StreamSets streaming processor code?

Can you share some examples or link on this?. The idea is to have 1 minute aggregation on real time streaming data and compute some KPI's. Please lets know how to achieve this by making use of any of the streaming framework (Flink/Kafka/Spark)?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-07-20 15:20:13 -0500

metadaddy gravatar image

At present only Spark Streaming is supported, via the Spark Evaluator processor. This tutorial explains how to build a simple Spark Transformer that executes in the evaluator.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-07-19 09:25:40 -0500

Seen: 524 times

Last updated: Jul 20 '18