Ask Your Question
0

SDC crashed unexpectedly after startup a while

asked 2018-05-23 20:55:17 -0500

casel.chen gravatar image

updated 2018-06-07 21:22:52 -0500

My sdc v3.0.3.0 works well at day but exits itself unexpectedly at night without any loads, anything wrong? Here is the sdc.log

2018-06-08 10:13:47,462 [user:admin] [pipeline:RISK_INVOCATION_HISTORY/RISKINVOCATIONHISTORY51d0f3a0-a7f7-4359-83c8-00031d09a8ad] [runner:0] [thread:ProductionPipelineRunnable-RISKINVOCATIONHISTORY51d0f3a0-a7f7-4359-83c8-00031d09a8ad-RISK_INVOCATION_HISTORY] WARN AppInfoParser - Error registering AppInfo mbean javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=consumer-4 at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:58) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:694) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:569) at com.streamsets.pipeline.kafka.impl.KafkaValidationUtil09.createTopicMetadataClient(KafkaValidationUtil09.java:151) at com.streamsets.pipeline.kafka.impl.KafkaValidationUtil09.getPartitionCount(KafkaValidationUtil09.java:58) at com.streamsets.pipeline.stage.destination.kafka.KafkaTargetConfig.getTopic(KafkaTargetConfig.java:672) at com.streamsets.pipeline.stage.destination.kafka.KafkaTarget.writeOneMessagePerRecord(KafkaTarget.java:202) at com.streamsets.pipeline.stage.destination.kafka.KafkaTarget.write(KafkaTarget.java:68) at com.streamsets.pipeline.configurablestage.DTarget.write(DTarget.java:34) at com.streamsets.datacollector.runner.StageRuntime$2.call(StageRuntime.java:253) at com.streamsets.datacollector.runner.StageRuntime$2.call(StageRuntime.java:237) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:195) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:264) at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:226) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.processPipe(ProductionPipelineRunner.java:751) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.lambda$runSourceLessBatch$3(ProductionPipelineRunner.java:779) at com.streamsets.datacollector.runner.PipeRunner.executeBatch(PipeRunner.java:132) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runSourceLessBatch(ProductionPipelineRunner.java:778) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runPollSource(ProductionPipelineRunner.java:550) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.run(ProductionPipelineRunner.java:375) at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:504) at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:112) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:74) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:754) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.retryOrStart(StandaloneRunner.java:371) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.onDataCollectorStart(StandaloneRunner.java:357) at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$onDataCollectorStart$0(AsyncRunner.java:105) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:227) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:223) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:227) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib ... (more)

edit retag flag offensive close merge delete

Comments

Yesterday night the same error happened again, "WebSocket connection time out". Does the sdc websocket connect to web page?

casel.chen gravatar imagecasel.chen ( 2018-05-24 21:26:54 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2018-05-29 04:03:15 -0500

Mufy gravatar image

That's the pipeline status/metrics/alerts collection communication that's seen to get affected. Outside the log event, how are you affected when this error is hit?

edit flag offensive delete link more

Comments

I found my SDC program exit, then checked the sdc.log and found above error log.

casel.chen gravatar imagecasel.chen ( 2018-05-29 08:39:55 -0500 )edit

It might help if you could attach your entire sdc.log in here so that I can take a quick look for you. I have a feeling the SDC "exit" and the WARN/ERROR seen here are two different events; not necessarily one causing the other.

Mufy gravatar imageMufy ( 2018-05-30 01:17:31 -0500 )edit

You can find my sdc.log file from here: https://gist.github.com/ChenShuai1981/593253bc741b7953203856ddbc622b6b And I supplement sdc startup script in above question

casel.chen gravatar imagecasel.chen ( 2018-05-30 09:09:56 -0500 )edit

Any suggestion to find the root SDC Crash cause? Total 6 pipelines. My VM is 2C8G. export SDC_JAVA_OPTS="-Xmx2g -Xms2g -server -XX:MaxDirectMemorySize=1g -XX:NewSize=256m -XX:MaxNewSize=256m -XX:-UseCompressedOops -XX:-OmitStackTraceInFastThrow -Doracle.jdbc.javaNetNio=true ${SDC_JAVA_OPTS}"

casel.chen gravatar imagecasel.chen ( 2018-06-04 20:51:53 -0500 )edit

How to do resource capacity estimation of SDC? Can I only adjust memory size by setting SDC_JAVA_OPTS in sdc-env.sh and sdcd-env.sh?

casel.chen gravatar imagecasel.chen ( 2018-06-07 20:49:58 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-05-23 20:55:17 -0500

Seen: 145 times

Last updated: Jun 07