Ask Your Question
1

'NoClassDefFoundError' occurred on Custom Stage

asked 2018-12-17 00:02:13 -0500

rockmkd gravatar image

I made very simple custom processor. It's maven dependency is below

<dependencies>
    <dependency>
        <groupId>com.streamsets</groupId>
        <artifactId>streamsets-datacollector-api</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>xerces</groupId>
        <artifactId>xercesImpl</artifactId>
        <version>1.12.0</version>
    </dependency>
</dependencies>

And Processor code is below

import org.apache.xerces.util.DOMUtil;

public class SimpleDOMProcessor extends RecordProcessor {

  public SimpleDOMProcessor(){
    try{
        DOMUtil domUtil = new DOMUtil();
    } catch(Exception ignored){
    }
  }
}

Packaging is success. I put the library in the user-libs and change sdc-security.policy for "all permission" After restarting SDC, 'NoClassDefFoundError' occurred when validating(also starting). ErorrLog is below

Caused by: java.lang.NoClassDefFoundError: org/apache/xerces/util/DOMUtil
  at com.sample.bigdata.sdc.stage.processor.SimpleDOMProcessor.<init>(SimpleDOMProcessor.java:31)
  at com.sample.bigdata.sdc.stage.processor.SimpleDOMDProcessor.createProcessor(SimpleDOMDProcessor.java:53)
  at com.streamsets.pipeline.api.base.configurablestage.DProcessor.createStage(DProcessor.java:30)
  at com.streamsets.pipeline.api.base.configurablestage.DStage.init(DStage.java:34)
  at com.streamsets.datacollector.runner.StageRuntime.lambda$init$0(StageRuntime.java:211)
  at com.streamsets.datacollector.util.LambdaUtil.withClassLoaderInternal(LambdaUtil.java:148)
  at com.streamsets.datacollector.util.LambdaUtil.withClassLoader(LambdaUtil.java:44)
  at com.streamsets.datacollector.runner.StageRuntime.init(StageRuntime.java:209)
  at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:123)
  at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:47)
  at com.streamsets.datacollector.runner.Pipeline.initPipe(Pipeline.java:402)
  at com.streamsets.datacollector.runner.Pipeline.lambda$init$0(Pipeline.java:392)
  at com.streamsets.datacollector.runner.PipeRunner.forEach(PipeRunner.java:166)
  at com.streamsets.datacollector.runner.Pipeline.init(Pipeline.java:390)
  at com.streamsets.datacollector.runner.Pipeline.validateConfigs(Pipeline.java:218)
  at com.streamsets.datacollector.runner.preview.PreviewPipeline.validateConfigs(PreviewPipeline.java:60)
  at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.validateConfigs(SyncPreviewer.java:144)

It'is very weired. I have made and use some complicated cusmtom processor but I have never met this situation.

If I change xercesImpl library to other simple library that's ok.(any problem is not occurred)

Is there any issue SDC with xercesImpl library?

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
1

answered 2018-12-17 14:49:31 -0500

metadaddy gravatar image

I don't see xercesImpl 1.12.0 anywhere on the web. Are you sure that's the correct version? I do see 2.12.0.

edit flag offensive delete link more

Comments

Thank you for answer. `2.12.0` is correct. Typo on writing this post. Anyway error is still occurring...

rockmkd gravatar imagerockmkd ( 2018-12-17 19:18:13 -0500 )edit

You might want to look at https://stackoverflow.com/questions/11677572/dealing-with-xerces-hell-in-java-maven - seems Xerces can be problematic

metadaddy gravatar imagemetadaddy ( 2018-12-17 19:25:48 -0500 )edit

@metadaddy thank you for resppnse. I have already read the post. I think since 2013 Xerces's maven probelm has gone. I made simple java application with Xeces,that's ok. thank you. i am digging this problem more.

rockmkd gravatar imagerockmkd ( 2018-12-17 20:20:02 -0500 )edit
0

answered 2018-12-19 04:04:53 -0500

rockmkd gravatar image

@metadaddy

I found some clue. api-classloader.proeprties describe some class namespace includes org.apache.xerces.

So I delete org.apache.xerces line and build streamsets-datacollector-bootstrap-3.4.3.jar by myself.(I am using version 3.4.3) Then the problem disappear!

It seems classloader issue. What can I do?

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-12-17 00:02:13 -0500

Seen: 49 times

Last updated: Dec 19 '18