Ask Your Question
1

Can i run streamset data collector run on Kubernetes through openshift.

asked 2018-12-11 16:04:36 -0600

trajan gravatar image

updated 2018-12-13 18:45:35 -0600

I can run the docker image on my local without any issues, but not able to run the same image when deployed through openshift. i know openshift have restriction on using the uid, https://docs.openshift.com/container-....

https://hub.docker.com/r/streamsets/d...

Java 1.8 detected; adding $SDC_JAVA8_OPTS of "-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Djdk.nio.maxCachedBufferSize=262144" to $SDC_JAVA_OPTS Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file /logs/gc.log due to Permission denied Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.streamsets.pipeline.BootstrapMain.main(BootstrapMain.java:307) Caused by: java.lang.RuntimeException: Could not create SDC ID file '/data/sdc.id': java.io.FileNotFoundException: /data/sdc.id (Permission denied) at com.streamsets.datacollector.main.StandaloneRuntimeInfo.getSdcId(StandaloneRuntimeInfo.java:82) at com.streamsets.datacollector.main.StandaloneRuntimeInfo.init(StandaloneRuntimeInfo.java:48) at com.streamsets.datacollector.main.RuntimeModule.provideRuntimeInfo(RuntimeModule.java:60) at com.streamsets.datacollector.main.RuntimeModule$$ModuleAdapter$ProvideRuntimeInfoProvidesAdapter.get(RuntimeModule$$ModuleAdapter.java:116) at com.streamsets.datacollector.main.RuntimeModule$$ModuleAdapter$ProvideRuntimeInfoProvidesAdapter.get(RuntimeModule$$ModuleAdapter.java:80) at dagger.internal.Linker$SingletonBinding.get(Linker.java:364)

so i did modify the Dokcerfile to support uid as entry point, and now getting a different error. And two fix the permission issue, i tried different commands,

Recommended from openshift RUN chgrp -R 0 /etc/sdc/form-realm.properties && \ chmod -R g=u /etc/sdc/form-realm.properties

RUN chmod 600 /etc/sdc/form-realm.properties

RUN chmod go-rwx /etc/sdc/form-realm.properties

RUN chown -R ${SDC_USER}:${SDC_USER} /etc/sdc/form-realm.properties

I get the either of the two error message

java.lang.RuntimeException: The permissions of the realm file '/etc/sdc/form-realm.properties' should be owner only at com.streamsets.datacollector.http.WebServerTask.validateRealmFile(WebServerTask.java:440)

or

java.lang.RuntimeException: java.nio.file.AccessDeniedException: /etc/sdc/form-realm.properties

edit retag flag offensive close merge delete

Comments

Can you post any error messages you see?

metadaddy gravatar imagemetadaddy ( 2018-12-11 18:43:51 -0600 )edit

2 Answers

Sort by ยป oldest newest most voted
1

answered 2019-01-21 16:21:32 -0600

trajan gravatar image

I was able to run it by coming up with custom sdc.properties files and overwrite the existing one on dockerfile. I set the http.realm.file.permission.check to false and used the latest version of streamsets tar from the archive to make it work

edit flag offensive delete link more
0

answered 2018-12-12 06:10:32 -0600

todd gravatar image

This portion appears as if it's permission related "Cannot open file /logs/gc.log due to Permission denied Exception"

edit flag offensive delete link more

Comments

1

i know it is happening due to permission issue, because Openshift doesn't support user id's and use arbitrary user id. But i did over come this error by making modification to the dockerfile.

trajan gravatar imagetrajan ( 2018-12-13 18:40:34 -0600 )edit

Can you please share the dockerfile?

AlokMajumder gravatar imageAlokMajumder ( 2019-09-18 11:16:00 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-12-11 16:04:36 -0600

Seen: 2,769 times

Last updated: Jan 21