Error Using MapReduce Executor In streamsets 3.7.0

asked 2019-01-31 03:16:05 -0500

seamfaces gravatar image

updated 2019-01-31 04:21:31 -0500

Hello Everyone,

I have setup streamsets 3.7.0 with the MapR v6.0.0-mep4 stage libraries, The MapR streams consumer, Producer and other components work correctly.

However when I try to attach the MapReduce executor to an Event produced from the MapR-FS destination, I get the following exception when previewing the pipeline

com.streamsets.datacollector.util.PipelineException: PREVIEW_0003 - Encountered error while previewing : java.lang.NoClassDefFoundError: org.apache.commons.configuration.Configuration     at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.validateConfigs(SyncPreviewer.java:166)    at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer$1.call(AsyncPreviewer.java:70)   at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)  at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable$$Lambda$113.000000001C00BCD0.call(Unknown Source)     at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)   at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)   at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)  at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable$$Lambda$113.000000001C00BCD0.call(Unknown Source)     at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)   at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)   at java.util.concurrent.FutureTask.run(FutureTask.java:266)     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)   at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100)     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:813) Caused by: java.lang.NoClassDefFoundError: org.apache.commons.configuration.Configuration  at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:38)     at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:36)   at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:145)     at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:159)  at com.streamsets.datacollector.security.MapRLoginUgiProvider.getLoginUgi(MapRLoginUgiProvider.java:48)     at com.streamsets.datacollector.security.HadoopSecurityUtil.getLoginUser(HadoopSecurityUtil.java:35)    at com.streamsets.pipeline.stage.destination.mapreduce.config.MapReduceConfig.init(MapReduceConfig.java:132)    at com.streamsets.pipeline.stage.destination.mapreduce.MapReduceExecutor.init(MapReduceExecutor.java:70)    at com.streamsets.pipeline.api.base.BaseStage.init(BaseStage.java:48)   at com.streamsets.pipeline.api.base.configurablestage.DStage.init(DStage.java:36)   at com.streamsets.datacollector.runner.StageRuntime.lambda$init$0(StageRuntime.java:211)    at com.streamsets.datacollector.runner.StageRuntime$$Lambda$143.0000000044743230.get(Unknown Source)    at com.streamsets.datacollector.util.LambdaUtil.withClassLoaderInternal(LambdaUtil.java:148)    at com.streamsets.datacollector.util.LambdaUtil.withClassLoader(LambdaUtil.java:44)     at com.streamsets.datacollector.runner.StageRuntime.init(StageRuntime.java:209)     at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:123)   at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:47)    at com.streamsets.datacollector.runner.Pipeline.initPipe(Pipeline.java:408)     at com.streamsets.datacollector.runner.Pipeline.lambda$init$0(Pipeline.java:397)    at com.streamsets.datacollector.runner.Pipeline$$Lambda$148.0000000044BF34F0.accept(Unknown Source)     at com.streamsets.datacollector.runner.PipeRunner.forEach ...
(more)
edit retag flag offensive close merge delete