Ask Your Question
0

START_ERROR: Error while sending pipeline status to Transformer instance

asked 2020-05-05 16:16:03 -0500

Deb gravatar image

Task - Read 2 CSV files, join on common key, write the output file.

  1. Installed Transformer, hadoop, spark on Ubuntu, accessing the pipelines using https://MY_IP:19630/
  2. Reading file using "File" and write it back using another "File"
  3. My files are stored in /drop folder

Initially got START_ERROR, certificate issue, then executed below 2 commands to import this URL's certificate

openssl x509 -in <(openssl s_client -connect MY_IP:19630 -prexit 2>/dev/null) -out ~/self.crt sudo keytool -importcert -file ~/self.crt -alias self_ss -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit

then now getting this:

Driver Error: Error while sending pipeline status to Transformer instance: https://MY_IP:19630, javax.ws.rs.ProcessingException: Error reading entity from input stream. javax.ws.rs.ProcessingException: Error reading entity from input stream. at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:889) at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:808) at org.glassfish.jersey.client.ClientResponse.readEntity(ClientResponse.java:326) at org.glassfish.jersey.client.InboundJaxrsResponse$1.call(InboundJaxrsResponse.java:115) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:228) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:419) at org.glassfish.jersey.client.InboundJaxrsResponse.runInScopeIfPossible(InboundJaxrsResponse.java:267) at org.glassfish.jersey.client.InboundJaxrsResponse.readEntity(InboundJaxrsResponse.java:112) at com.streamsets.datatransformer.dag.runner.DataTransformerRunner.callbackLauncher(DataTransformerRunner.java:438) at com.streamsets.datatransformer.dag.runner.DataTransformerRunner.sendStatusToLauncher(DataTransformerRunner.java:378) at com.streamsets.datatransformer.dag.runner.DataTransformerRunner.start(DataTransformerRunner.java:201) at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:151) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Invalid Http response at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1950) at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1945) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java ...

(more)
edit retag flag offensive close merge delete

3 Answers

Sort by ยป oldest newest most voted
0

answered 2020-06-19 10:46:05 -0500

Deb gravatar image

updated 2020-06-24 15:51:10 -0500

Alright, so I did more research, and here is the conclusion:

  1. @jeff @J3 - I am using http
  2. If I start transformer manually, I do not get any error and pipeline just works fine.
  3. If I start automatically with init.d, it throws the error

I understand that for init.d it uses from /etc/transformer/transformer.properties and for manual starts, it reads from $TRANSFORMER_DIST/etc, in my case it is /opt/streamsets/streamsets-transformer-3.13.0/etc

and both places base http url is set properly like this:

transformer.base.http.url=http://MY_IP:19630

edit flag offensive delete link more
0

answered 2020-05-06 16:25:23 -0500

jeff gravatar image

If you actually intend to use https, then did you follow these instructions for configuring the truststore file? From the command you pasted, it seems you updated the JVM's embedded store, which is not what's needed.

edit flag offensive delete link more
0

answered 2020-05-06 15:19:56 -0500

J3 gravatar image

I notice the URL you are using is https://[YOUR IP]:19630, but I don't think you configured it for HTTPS. Correct? (Note, out-of-box, Transformer is not configured for HTTPS.)

If so, try HTTP://[YOUR IP]:19630.

Stay safe!! J3

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-05-05 16:16:03 -0500

Seen: 95 times

Last updated: Jun 24