We are getting error on using google NLP on spark processor, i think it is due to conflicting Guava.jar.

asked 2019-05-03 03:05:31 -0600

amit gravatar image

updated 2019-05-03 09:57:36 -0600

metadaddy gravatar image

We are getting error on using google NLP on spark processor, i think it is due to conflicting Guava.jar. but there are lots of different versions of this jar and not able to identify which one to replace. in our java code which is using google NLP we are using guava.20.0.jar, still we are getting the following error:

Pipeline Status: RUNNING_ERROR: com.streamsets.pipeline.api.StageException: SPARK_07 - Spark job failed with error: Job aborted due to stage failure: Task 3 in stage 1.0 failed 1 times, most recent failure: Lost task 3.0 in stage 1.0 (TID 7, localhost, executor driver): java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;CLjava/lang/Object;)V
at io.grpc.Metadata$Key.validateName(Metadata.java:629)
at io.grpc.Metadata$Key.(Metadata.java:637)
at io.grpc.Metadata$Key.(Metadata.java:567)
at io.grpc.Metadata$AsciiKey.(Metadata.java:742)
at io.grpc.Metadata$AsciiKey.(Metadata.java:737)
at io.grpc.Metadata$Key.of(Metadata.java:593)
at io.grpc.Metadata$Key.of(Metadata.java:589)
at com.google.api.gax.grpc.GrpcHeaderInterceptor.(GrpcHeaderInterceptor.java:60)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:180)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:164)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:156)
at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:157)
at com.google.cloud.language.v1.stub.GrpcLanguageServiceStub.create(GrpcLanguageServiceStub.java:130)
at com.google.cloud.language.v1.stub.LanguageServiceStubSettings.createStub(LanguageServiceStubSettings.java:139)
at com.google.cloud.language.v1.LanguageServiceClient.(LanguageServiceClient.java:132)
at com.google.cloud.language.v1.LanguageServiceClient.create(LanguageServiceClient.java:113)
at com.google.nlp.demo.Analyze.analyzeSentimentText(Analyze.java:53)
at com.google.nlp.demo.CustomTransformer$1.call(CustomTransformer.java:52)
at com.google.nlp.demo.CustomTransformer$1.call(CustomTransformer.java:41)
at org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1.apply(JavaPairRDD.scala:1040)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at scala.collection.AbstractIterator.to(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:936)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:936)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2071)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2071)
at org.apache.spark.scheduler.ResultTask.runTask ...
(more)
edit retag flag offensive close merge delete