Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Can i run streamset data collector run on Kubernetes through openshift.

I can run the docker image on my local without any issues, but not able to run the same image when deployed through openshift.

https://hub.docker.com/r/streamsets/datacollector/~/dockerfile/

Can i run streamset data collector run on Kubernetes through openshift.

I can run the docker image on my local without any issues, but not able to run the same image when deployed through openshift.openshift. i know openshift have restriction on using the uid, https://docs.openshift.com/container-platform/3.6/creating_images/guidelines.html.

https://hub.docker.com/r/streamsets/datacollector/~/dockerfile/

Java 1.8 detected; adding $SDC_JAVA8_OPTS of "-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Djdk.nio.maxCachedBufferSize=262144" to $SDC_JAVA_OPTS Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file /logs/gc.log due to Permission denied Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.streamsets.pipeline.BootstrapMain.main(BootstrapMain.java:307) Caused by: java.lang.RuntimeException: Could not create SDC ID file '/data/sdc.id': java.io.FileNotFoundException: /data/sdc.id (Permission denied) at com.streamsets.datacollector.main.StandaloneRuntimeInfo.getSdcId(StandaloneRuntimeInfo.java:82) at com.streamsets.datacollector.main.StandaloneRuntimeInfo.init(StandaloneRuntimeInfo.java:48) at com.streamsets.datacollector.main.RuntimeModule.provideRuntimeInfo(RuntimeModule.java:60) at com.streamsets.datacollector.main.RuntimeModule$$ModuleAdapter$ProvideRuntimeInfoProvidesAdapter.get(RuntimeModule$$ModuleAdapter.java:116) at com.streamsets.datacollector.main.RuntimeModule$$ModuleAdapter$ProvideRuntimeInfoProvidesAdapter.get(RuntimeModule$$ModuleAdapter.java:80) at dagger.internal.Linker$SingletonBinding.get(Linker.java:364)

Can i run streamset data collector run on Kubernetes through openshift.

I can run the docker image on my local without any issues, but not able to run the same image when deployed through openshift. i know openshift have restriction on using the uid, https://docs.openshift.com/container-platform/3.6/creating_images/guidelines.html.

https://hub.docker.com/r/streamsets/datacollector/~/dockerfile/

Java 1.8 detected; adding $SDC_JAVA8_OPTS of "-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Djdk.nio.maxCachedBufferSize=262144" to $SDC_JAVA_OPTS Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file /logs/gc.log due to Permission denied Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.streamsets.pipeline.BootstrapMain.main(BootstrapMain.java:307) Caused by: java.lang.RuntimeException: Could not create SDC ID file '/data/sdc.id': java.io.FileNotFoundException: /data/sdc.id (Permission denied) at com.streamsets.datacollector.main.StandaloneRuntimeInfo.getSdcId(StandaloneRuntimeInfo.java:82) at com.streamsets.datacollector.main.StandaloneRuntimeInfo.init(StandaloneRuntimeInfo.java:48) at com.streamsets.datacollector.main.RuntimeModule.provideRuntimeInfo(RuntimeModule.java:60) at com.streamsets.datacollector.main.RuntimeModule$$ModuleAdapter$ProvideRuntimeInfoProvidesAdapter.get(RuntimeModule$$ModuleAdapter.java:116) at com.streamsets.datacollector.main.RuntimeModule$$ModuleAdapter$ProvideRuntimeInfoProvidesAdapter.get(RuntimeModule$$ModuleAdapter.java:80) at dagger.internal.Linker$SingletonBinding.get(Linker.java:364)

so i did modify the Dokcerfile to support uid as entry point, and now getting a different error. And two fix the permission issue, i tried different commands,

Recommended from openshift RUN chgrp -R 0 /etc/sdc/form-realm.properties && \ chmod -R g=u /etc/sdc/form-realm.properties

RUN chmod 600 /etc/sdc/form-realm.properties

RUN chmod go-rwx /etc/sdc/form-realm.properties

RUN chown -R ${SDC_USER}:${SDC_USER} /etc/sdc/form-realm.properties

I get the either of the two error message

java.lang.RuntimeException: The permissions of the realm file '/etc/sdc/form-realm.properties' should be owner only at com.streamsets.datacollector.http.WebServerTask.validateRealmFile(WebServerTask.java:440)

or

java.lang.RuntimeException: java.nio.file.AccessDeniedException: /etc/sdc/form-realm.properties