Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Kinesis producer issues

I am using streamSets 3.13.0.

and getting the errors while using kinesis producer

com.amazonaws.services.kinesis.producer.DaemonException: The child process has been shutdown and can no longer accept messages. at com.amazonaws.services.kinesis.producer.Daemon.add(Daemon.java:176) at com.amazonaws.services.kinesis.producer.KinesisProducer.addUserRecord(KinesisProducer.java:536) at com.amazonaws.services.kinesis.producer.KinesisProducer.addUserRecord(KinesisProducer.java:349) at com.streamsets.pipeline.stage.destination.kinesis.KinesisTarget.write(KinesisTarget.java:240) at com.streamsets.pipeline.api.base.configurablestage.DTarget.write(DTarget.java:34) at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:303) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:244) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:311) at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:220) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.processPipe(ProductionPipelineRunner.java:850) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.lambda$executeRunner$3(ProductionPipelineRunner.java:894) at com.streamsets.datacollector.runner.PipeRunner.acceptConsumer(PipeRunner.java:221) at com.streamsets.datacollector.runner.PipeRunner.executeBatch(PipeRunner.java:142) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.executeRunner(ProductionPipelineRunner.java:893) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runSourceLessBatch(ProductionPipelineRunner.java:871) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.processBatch(ProductionPipelineRunner.java:505) at com.streamsets.datacollector.runner.StageRuntime$3.run(StageRuntime.java:383) at java.security.AccessController.doPrivileged(Native Method) at com.streamsets.datacollector.runner.StageRuntime.processBatch(StageRuntime.java:379) at com.streamsets.datacollector.runner.StageContext.processBatch(StageContext.java:284) at com.streamsets.pipeline.stage.devtest.RandomDataGeneratorSource$GeneratorRunnable.run(RandomDataGeneratorSource.java:263) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

and

when job start to retry getting this error:

com.amazonaws.services.kinesis.producer.IrrecoverableError: Error starting child process at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:537) at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:468) at com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63) at com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Cannot run program "/tmp/amazon-kinesis-producer-native-binaries/kinesis_producer_D626AFB58CB68D8C81E7F8671FE0048978DAA5B6": error=13, Permission denied at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:466) ... 5 more Caused by: java.io.IOException: error=13, Permission denied at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:247) at java.lang.ProcessImpl.start(ProcessImpl.java:134) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ... 6 more