streamsets cli to import the pipeline with ssl in Docker

asked 2020-01-15 16:44:54 -0600

Prometheous2019 gravatar image

Hi All,

I have created a base StreamSets ([https://github.com/streamsets/datacol...]) docker image. Which runs fine and well without SSL/certificate using my ldap authentication. So, inside the container I execute below command to upload pipeline in JSON.

streamsets cli -U http://localhost:<port> -u "ldap-username"  -p "ldap-password"  "rest of command to upload the pipeline"

Now, I have to enable HTTPS. So, as per the documentation, hence I updated, sdc.properties file for http port as -1 and https port as "my port number" and edited my keystore and password file name . And rebuild my image and run the container, but without the pipeline . So that I can verify the settings are right and all certificates from SS like keystore.jks and password files are created and can login using my ldap credentials. and in the container log it also shows, https://conatinerid:port (pay attention to https) as URI is up and running. of course my frontend shows little warning and after accepting the not secure button;I can go and login to my streamset conatiner.

Now if I come back to container and run

streamsets cli -U https://localhost:<port> -u "ldap-username"  -p "ldap-password"  "rest of command to upload the pipeline"

then I get javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException

Then I have updated: export SDC_CLI_JAVA_OPTS="-Djavax.net.ssl.trustStore=${SDC_CONF}/keystore.jks -Djavax.net.ssl.trustStorePassword=$(cat ${SDC_CONF}/keystore-password.txt)"

after then I re-ran the cli command I still get the same error. (but didn't restart the streamset)

Is there any way I can say ignore ssl and upload the pipeline un-securely, like curl -k. Also, I was looking/exploring options using python SDK, couldn't find any argument to say ssl verfication = off . Is there any way I can upload pipeline using my ldap credentials like I was able to do without https

edit retag flag offensive close merge delete

Comments

Did you verify that ${SDC_CONF} was as expected before you ran the cli command, and that the files are there on the machine on which you're running the command?

metadaddy gravatar imagemetadaddy ( 2020-01-15 22:59:45 -0600 )edit

yes....I did.

Prometheous2019 gravatar imagePrometheous2019 ( 2020-01-15 23:57:27 -0600 )edit