Unable to access remote server ppk from streamsets.

asked 2019-01-07 01:30:16 -0600

sandeep110200 gravatar image

I used to connect to one of the VM from my Streamsets for accessing the files through SFTP using ppk. All my pipelines are able to access the files through SFTP but when i stopped any pipeline and try to connect to file system the pipeline is giving the exception like "REMOTE_18 - Error accessing remote directory: Could not connect to SFTP server at "sftp://user@myserverip/". Below is the stacktrace for the same.

com.streamsets.datacollector.runner.PipelineRuntimeException: CONTAINER_0800 - Pipeline 'CLIAVQSFTPTESTINGcopyb5b3d2af-750a-4931-a076-d2e0c456ef34' validation error : REMOTE_18 - Error accessing remote directory: Could not connect to SFTP server at "sftp://username@hostname/". at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:128) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:75) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:724) at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:151) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

edit retag flag offensive close merge delete

Comments

Are you saying that you have several pipelines accessing SFTP successfully, you stop one, then the others stop working?

metadaddy gravatar imagemetadaddy ( 2019-01-08 11:18:15 -0600 )edit

I have several running pipelines accessing SFTP but if stop one and try to run again its not able to connect to the vm using sftp.

sandeep110200 gravatar imagesandeep110200 ( 2019-01-10 01:39:15 -0600 )edit