Ask Your Question

How can we collect data from different clouds?

asked 2017-12-13 02:44:38 -0600

anonymous user


updated 2018-02-07 11:16:35 -0600

metadaddy gravatar image

If my origin and destination are on different clouds how can we collect data? My origin is MariaDB and destination is HDFS.

edit retag flag offensive close merge delete


Which cloud providers are you using specifically?

metadaddy gravatar imagemetadaddy ( 2017-12-13 11:03:12 -0600 )edit

Streamsets uses DigitalOcean and origin,destination uses Azure. Will that be an issue?

Shruthi gravatar imageShruthi ( 2017-12-13 23:50:03 -0600 )edit

1 Answer

Sort by » oldest newest most voted

answered 2018-02-07 11:16:09 -0600

metadaddy gravatar image

updated 2018-02-07 11:18:14 -0600

As long as you can get remote access to at least one of the clouds, this shouldn't be a problem.

For example, if StreamSets Data Collector is reading data from MariaDB and writing to Azure, it could make either a local or remote connection to read the data (you would have to ensure that network access was appropriately configured), and use the Azure Data Lake Store destination to write the data.

Another possibility would be to run SDC on Azure HDInsight, in which case you would definitely need to make a remote connection to MariaDB if it were on a different cloud.

edit flag offensive delete link more


Thank you for your response

Shruthi gravatar imageShruthi ( 2018-02-07 22:57:30 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-12-13 02:44:38 -0600

Seen: 175 times

Last updated: Feb 07 '18