kafka consumer fails to read avro data

asked 2019-02-26 22:41:07 -0500

spider8686 gravatar image

I have a schema in schema registry. i sent data to producer with avro schema but consumer failing to read data from producer. not sure kafka-avro-console-producer is not getting data. .log file doesn't have data in binary format.

error from consumer side.

com.streamsets.pipeline.api.base.OnRecordErrorException: KAFKA_37 - Cannot parse record from message 'a::0::132017': com.streamsets.pipeline.lib.parser.DataParserException: DATA_PARSER_02 - Parser error: 'org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -40' at com.streamsets.pipeline.stage.origin.kafka.BaseKafkaSource.processKafkaMessageDefault(BaseKafkaSource.java:251) at com.streamsets.pipeline.stage.origin.kafka.StandaloneKafkaSource.produce(StandaloneKafkaSource.java:76) at com.streamsets.pipeline.api.base.configurablestage.DSource.produce(DSource.java:38) at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:283) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:235) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:298) at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:244) at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.runPollSource(PreviewPipelineRunner.java:321) at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.run(PreviewPipelineRunner.java:218) at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:539) at com.streamsets.datacollector.runner.preview.PreviewPipeline.run(PreviewPipeline.java:51) at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:217) at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer.lambda$start$0(AsyncPreviewer.java:95) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: com.streamsets.pipeline.lib.parser.DataParserException: DATA_PARSER_02 - Parser error: 'org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -40' at com.streamsets.pipeline.lib.parser.WrapperDataParserFactory$WrapperDataParser.normalizeException(WrapperDataParserFactory.java:141) at com.streamsets.pipeline.lib.parser.WrapperDataParserFactory$WrapperDataParser.parse(WrapperDataParserFactory.java:101) at com.streamsets.pipeline.stage.origin.kafka.BaseKafkaSource.processKafkaMessageDefault(BaseKafkaSource.java:234) ... 25 more Caused by: org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -40 at org.apache.avro.io.BinaryDecoder.doReadBytes(BinaryDecoder.java:336) at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:263) at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201) at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:363) at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:355) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:157) at org.apache.avro.generic ... (more)

edit retag flag offensive close merge delete