Ask Your Question

SDC Support for Windows OS

asked 2017-12-14 09:02:55 -0600

prithumit gravatar image

updated 2017-12-15 15:34:39 -0600

metadaddy gravatar image

We are evaluating StreamSets for a FinTech solution where we think SDC could be the best fit for solution related to dataflow architecture and connector layer between heterogeneous data. I am aware that SDC is not officially supported on Windows. But we have a great need to have a solution for Windows OS too and it would have been picture perfect solution is SDC could be installed in Windows OS. With regards to that I have the following queries

  • Though not officially supported, but can StreamSets DC run on Windows? Are there work around that can be done to make it running in Windows?
  • Are there any examples or document settings somewhere which I can refer in this regards? there may not be an installer but can SDC be configured in Windows manually
  • Is there any known challenges or reason why SDC is not supported in Windows?
  • Is there a plan in near future to Support Windows OS also?

I am really hopeful that there could be a workaround solution for Windows for this awesome technology. Awaiting reply on this. Please let me know

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2017-12-15 16:13:45 -0600

metadaddy gravatar image

updated 2017-12-15 16:20:26 -0600

For the purposes of this answer, I'll assume you've considered and rejected the possibility of running the SDC Docker image with Docker for Windows.

Although StreamSets Data Collector is a Java app, which in theory can run 'anywhere', it relies on several bash shell scripts, for example bin/streamsets and libexec/_sdc set up the environment and run the actual Java app.

There are several possible ways forward:

Of course, even the app starts, there may well be code changes required due to assumptions in the code that do not hold true on Windows.

The main reason none of the above has been tried is that we haven't seen much demand for running SDC on Windows. Most big data processing takes place on Linux - even Azure HDInsight is Linux-based.

The whole product is open source, of course, so you are very welcome to take a look at running it on Windows, and even submit pull requests with any necessary changes.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-12-14 09:02:55 -0600

Seen: 4,370 times

Last updated: Dec 15 '17