Ask Your Question

Can StreamSets Data Collector ingest data from ActiveMQ?

asked 2018-03-13 14:07:41 -0500

tpsv gravatar image

updated 2018-03-13 15:37:08 -0500

metadaddy gravatar image

We want to read from ActiveMQ queue and then write that data to HBase via Phoenix. Is that something we can achieve with StreamSets Data Collector?

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted

answered 2018-03-13 15:43:20 -0500

metadaddy gravatar image

Yes - you should be able to get something working.

ActiveMQ supports MQTT and AMQP; Data Collector has an MQTT Subscriber Origin, and also a RabbitMQ Consumer Origin which uses AMQP, so that may also work.

Data Collector has an HBase Destination, so you can write data directly, without going via Phoenix.

edit flag offensive delete link more


Thanks. That helps. As Phoenix supports secondary indexing and HBase does not, I was wondering if read/write to HBase using Data Collector will be as efficient or better than using Phoenix in-between?

tpsv gravatar imagetpsv ( 2018-03-14 08:48:59 -0500 )edit

answered 2018-03-13 15:40:58 -0500

kirti gravatar image

updated 2018-03-13 15:44:06 -0500

metadaddy gravatar image

Yes StreamSets can ingest data from ActiveMQ. Take a look here:

and also

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-03-13 14:07:41 -0500

Seen: 116 times

Last updated: Mar 13 '18