Ask Your Question
0

MapR cluster batch mode mysql-connector-java-*-bin.jar not found

asked 2017-12-17 22:41:52 -0500

davidha gravatar image

updated 2017-12-18 13:59:54 -0500

metadaddy gravatar image

Hello, I am creating a pipeline following this document in my MapR cluster: https://streamsets.com/documentation/...

My pipeline is very simple, origin as "MapR FS" to destination of "Local FS", however, when I run the pipeline, the following error pop up while i did found mysql-connector-java-*-bin.jar in the directory. What might be the root cause?

Pipeline Status: START_ERROR: Unexpected error starting pipeline: java.lang.RuntimeException: ERROR: Serializing classpath: 'java.io.FileNotFoundException: /opt/streamsets-datacollector-2.7.2.0/streamsets-libs/streamsets-datacollector-mapr_5_2-lib/lib/mysql-connector-java-*-bin.jar (No such file or directory)'

Thank you! David

edit retag flag offensive close merge delete

Comments

You might get that error when SDC doesn't have the proper permissions, I assume that SDC is the owner as you followed the document. Give the rwx permissions to the sdc user and see.

Roh gravatar imageRoh ( 2017-12-22 12:50:24 -0500 )edit

Hello, I am using another user to start streamsets with commandline /bin/streamsets dc. For instance, 'usera'. However, even I change the files inside /opt/streamsets-datacollector-2.7.2.0/streamsets-libs/streamsets-datacollector-mapr_5_2-lib/lib/ to 777 permission for testing, the same error still

davidha gravatar imagedavidha ( 2018-01-03 00:22:37 -0500 )edit

Please confirm the output of the command below /opt/streamsets-datacollector-2.7.2.0/streamsets-libs/streamsets-datacollector-mapr_5_2-lib/lib|grep mysql and what MEP version do you've installed on top of MapR 5.2?

rupal gravatar imagerupal ( 2018-01-06 17:10:51 -0500 )edit

The command output is : -bash: /opt/streamsets-datacollector-2.7.2.0/streamsets-libs/streamsets-datacollector-mapr_5_2-lib/lib: Is a directory I try to list out what directory instead with ls lib/mysql* : lib/mysql-connector-java-5.1.25.jar lib/mysql-connector-java-*-bin.jar. Required files exist.

davidha gravatar imagedavidha ( 2018-01-06 22:32:04 -0500 )edit

MEP version:3.0.2

davidha gravatar imagedavidha ( 2018-01-06 22:32:46 -0500 )edit

2 Answers

Sort by ยป oldest newest most voted
0

answered 2018-02-27 02:38:38 -0500

davidha gravatar image

Is this certified as a bug? Seems not working under any situation for the MapR FS origin.

edit flag offensive delete link more
0

answered 2018-01-06 01:22:52 -0500

Selva gravatar image

Hi,

I do get an error "Unexpected error starting pipeline: java.lang.RuntimeException: ERROR: Serializing classpath: 'java.io.FileNotFoundException: /opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-mapr_6_0-lib/lib/mysql-connector-java-*-bin.jar (No such file or directory)''

But the file exists as shown in the below path.

/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-mapr_6_0-lib/lib/mysql-connector-java-*-bin.jar

ava.lang.RuntimeException: ERROR: Serializing classpath: 'java.io.FileNotFoundException: /opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-mapr_6_0-lib/lib/mysql-connector-java--bin.jar (No such file or directory)' at com.streamsets.datacollector.cluster.ClusterProviderImpl.startPipelineInternal(ClusterProviderImpl.java:704) at com.streamsets.datacollector.cluster.ClusterProviderImpl.startPipeline(ClusterProviderImpl.java:500) at com.streamsets.datacollector.execution.cluster.ClusterHelper.submit(ClusterHelper.java:88) at com.streamsets.datacollector.execution.runner.cluster.ClusterRunner.doStart(ClusterRunner.java:876) at com.streamsets.datacollector.execution.runner.cluster.ClusterRunner.start(ClusterRunner.java:441) at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:152) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:227) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:223) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:227) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:223) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.FileNotFoundException: /opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-mapr_6_0-lib/lib/mysql-connector-java--bin.jar (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at com.streamsets.datacollector.cluster.TarFileCreator.addClasspath(TarFileCreator.java:97) at com.streamsets.datacollector.cluster.TarFileCreator.addLibrary(TarFileCreator.java:85) at com.streamsets.datacollector.cluster.TarFileCreator.createLibsTarGz(TarFileCreator.java:54) at com.streamsets.datacollector.cluster.ClusterProviderImpl.startPipelineInternal(ClusterProviderImpl.java:694) Selva

edit flag offensive delete link more

Comments

mysql-connector-java-*-bin.jar is a symbolic link to /opt/mapr/hive/hive-2.1/lib/mysql-connector-java-*-bin.jar which is a symbolic link to /opt/mapr/lib/mysql-connector-java-5.1.25.jar !

Selva gravatar imageSelva ( 2018-01-10 03:17:25 -0500 )edit
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2017-12-17 22:41:52 -0500

Seen: 78 times

Last updated: Feb 27