Ask Your Question
1

Can StreamSets Data Collector ingest data from ActiveMQ?

asked 2018-03-13 14:07:41 -0500

tpsv gravatar image

updated 2018-03-13 15:37:08 -0500

metadaddy gravatar image

We want to read from ActiveMQ queue and then write that data to HBase via Phoenix. Is that something we can achieve with StreamSets Data Collector?

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
0

answered 2018-03-13 15:43:20 -0500

metadaddy gravatar image

Yes - you should be able to get something working.

ActiveMQ supports MQTT and AMQP; Data Collector has an MQTT Subscriber Origin, and also a RabbitMQ Consumer Origin which uses AMQP, so that may also work.

Data Collector has an HBase Destination, so you can write data directly, without going via Phoenix.

edit flag offensive delete link more

Comments

Thanks. That helps. As Phoenix supports secondary indexing and HBase does not, I was wondering if read/write to HBase using Data Collector will be as efficient or better than using Phoenix in-between?

tpsv gravatar imagetpsv ( 2018-03-14 08:48:59 -0500 )edit
0

answered 2018-03-13 15:40:58 -0500

kirti gravatar image

updated 2018-03-13 15:44:06 -0500

metadaddy gravatar image

Yes StreamSets can ingest data from ActiveMQ. Take a look here: https://streamsets.com/documentation/...

and also https://streamsets.com/blog/ingesting...

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-03-13 14:07:41 -0500

Seen: 25 times

Last updated: Mar 13