SDC_JAVA_OPTS not configuring Java properties

asked 2019-04-26 10:14:31 -0500

Roman gravatar image

updated 2019-05-02 09:05:06 -0500

I am using StreamSets Data Collector and am trying to configure TLS. I am following this guide: https://streamsets.com/documentation/...

In Step 3, it says to configure a few sdc.properties via environment variables and some Java properties in the same way. The SDC properties are being set properly. However, the Java properties are never being set. I am putting them into an environment variable "SDC_JAVA_OPTS". The env var is populated but the values are never set in the Java properties file "$JAVA_HOME/jre/lib/management/management.properties". This is causing Streamsets to crash with this error from Java: "java.lang.RuntimeException: java.io.IOException: Keystore was tampered with, or password was incorrect, calling stopTask() and going into ERROR"

Note: I am running SDC 3.8.0 in Docker container inside Kubernetes deploying with Helm.

Update 5/1: I was able to turn on verbose mode and it appears sdc-env.sh is being run and any values I set to SDC_JAVA_OPTS are getting into the env at startup. The problem still remains that my values are not being set in the java management.properties file. You can see below a part of the verbose output showing the JAVA OPTS at startup. I have added -Djavax.net.ssl.trustStore and -Djavax.net.ssl.trustStorePassword via K8S environment variable. Verbose output: "JAVA OPTS : -Djava.security.manager -Djava.security.policy=file:///etc/sdc/sdc-security.policy -Xmx1024m -Xms1024m -server -XX:-OmitStackTraceInFastThrow -Djavax.net.ssl.trustStore=/data/security/sdc-company.truststore.jks -Djavax.net.ssl.trustStorePassword=** -Dsdc.di......"

Update 5/2: It appears that my keystore/truststore files and passwords are input correctly (no errors). Unfortunately once I set the HTTPS port it brakes my UI. I think it's just a port issue with my k8s service. I think this can be closed now, thank you for your help.

Any help would be appreciated.

Thank you.

edit retag flag offensive close merge delete

Comments

1

What type of SDC installation do you have? Are you sure you set the SDC_JAVA_OPTS in the correct place, for your installation type?

jeff gravatar imagejeff ( 2019-04-26 13:11:15 -0500 )edit

I am deploying the SDC 3.8.0 container with Helm to a K8S cluster. I am setting things like SDC_CONF_HTTPS_KEYSTORE_PATH and SDC_JAVA_OPTS in my Helm chart.

Roman gravatar imageRoman ( 2019-04-26 13:23:46 -0500 )edit

I also noticed in this file: $SDC_DIST/libexec/sdc-env.sh that it is doing an export on SDC_JAVA_OPTS `export SDC_JAVA_OPTS="-Xmx1024m -Xms102". However, unless I set SDC_JAVA_OPTS. This var is empty/not set. It’s almost like sdc-env.sh is not being run on sdc start up.

Roman gravatar imageRoman ( 2019-04-26 13:24:31 -0500 )edit

From your link above: "Data Collector loads the value of the version-specific environment variable and adds it to the SDC_JAVA_OPTS environment variable". This is not being set by default. This is what I was referring to in my last comment. ^

Roman gravatar imageRoman ( 2019-04-26 13:46:20 -0500 )edit