Ask Your Question

Can i run streamset data collector run on Kubernetes through openshift.

asked 2018-12-11 16:04:36 -0500

trajan gravatar image

updated 2018-12-13 18:45:35 -0500

I can run the docker image on my local without any issues, but not able to run the same image when deployed through openshift. i know openshift have restriction on using the uid,

Java 1.8 detected; adding $SDC_JAVA8_OPTS of "-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Djdk.nio.maxCachedBufferSize=262144" to $SDC_JAVA_OPTS Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file /logs/gc.log due to Permission denied Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( at sun.reflect.DelegatingMethodAccessorImpl.invoke( at java.lang.reflect.Method.invoke( at com.streamsets.pipeline.BootstrapMain.main( Caused by: java.lang.RuntimeException: Could not create SDC ID file '/data/': /data/ (Permission denied) at com.streamsets.datacollector.main.StandaloneRuntimeInfo.getSdcId( at com.streamsets.datacollector.main.StandaloneRuntimeInfo.init( at com.streamsets.datacollector.main.RuntimeModule.provideRuntimeInfo( at com.streamsets.datacollector.main.RuntimeModule$$ModuleAdapter$ProvideRuntimeInfoProvidesAdapter.get(RuntimeModule$$ at com.streamsets.datacollector.main.RuntimeModule$$ModuleAdapter$ProvideRuntimeInfoProvidesAdapter.get(RuntimeModule$$ at dagger.internal.Linker$SingletonBinding.get(

so i did modify the Dokcerfile to support uid as entry point, and now getting a different error. And two fix the permission issue, i tried different commands,

Recommended from openshift RUN chgrp -R 0 /etc/sdc/ && \ chmod -R g=u /etc/sdc/

RUN chmod 600 /etc/sdc/

RUN chmod go-rwx /etc/sdc/

RUN chown -R ${SDC_USER}:${SDC_USER} /etc/sdc/

I get the either of the two error message

java.lang.RuntimeException: The permissions of the realm file '/etc/sdc/' should be owner only at com.streamsets.datacollector.http.WebServerTask.validateRealmFile(


java.lang.RuntimeException: java.nio.file.AccessDeniedException: /etc/sdc/

edit retag flag offensive close merge delete


Can you post any error messages you see?

metadaddy gravatar imagemetadaddy ( 2018-12-11 18:43:51 -0500 )edit

2 Answers

Sort by ยป oldest newest most voted

answered 2019-01-21 16:21:32 -0500

trajan gravatar image

I was able to run it by coming up with custom files and overwrite the existing one on dockerfile. I set the http.realm.file.permission.check to false and used the latest version of streamsets tar from the archive to make it work

edit flag offensive delete link more

answered 2018-12-12 06:10:32 -0500

todd gravatar image

This portion appears as if it's permission related "Cannot open file /logs/gc.log due to Permission denied Exception"

edit flag offensive delete link more



i know it is happening due to permission issue, because Openshift doesn't support user id's and use arbitrary user id. But i did over come this error by making modification to the dockerfile.

trajan gravatar imagetrajan ( 2018-12-13 18:40:34 -0500 )edit

Can you please share the dockerfile?

AlokMajumder gravatar imageAlokMajumder ( 2019-09-18 11:16:00 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-12-11 16:04:36 -0500

Seen: 3,727 times

Last updated: Jan 21 '19