Ask Your Question
1

Cannot start SDC 3.9.0

asked 2019-06-26 02:53:44 -0500

Nightwatch gravatar image

updated 2019-06-27 03:52:31 -0500

Hi,

I've upgraded SDC from 3.8.2 to 3.9.0 but after that the service doesn't start. I can see on gc.log that maybe something related to java memory issues.

Java HotSpot(TM) 64-Bit Server VM (25.201-b09) for linux-amd64 JRE (1.8.0_201-b09), built on Dec 15 2018 11:35:59 by "java_re" with gcc 7.3.0
Memory: 4k page, physical 8009556k(7749060k free), swap 4194300k(4194300k free)
CommandLine flags: -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/sdc/sdc_heapdump_1561534689.hprof -XX:InitialHeapSize=1073741824 -XX:MaxHeapSize=1073741824 -XX:MaxNewSize=174485504 -XX:MaxTenuringThreshold=6 -XX:NewSize=174485504 -XX:OldPLABSize=16 -XX:OldSize=348971008 -XX:-OmitStackTraceInFastThrow -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC 
2019-06-26T09:38:50.464+0200: 37.826: [GC (Allocation Failure) 2019-06-26T09:38:50.465+0200: 37.826: [ParNew: 136320K->12245K(153344K), 0.0863098 secs] 136320K->12245K(1031552K), 0.0864338 secs] [Times: user=0.03 sys=0.00, real=0.09 secs] 
2019-06-26T09:39:23.982+0200: 71.344: [GC (Allocation Failure) 2019-06-26T09:39:23.982+0200: 71.344: [ParNew: 148565K->7820K(153344K), 0.1149114 secs] 148565K->14721K(1031552K), 0.1150215 secs] [Times: user=0.07 sys=0.00, real=0.12 secs] 
2019-06-26T09:39:50.059+0200: 97.420: [GC (Allocation Failure) 2019-06-26T09:39:50.059+0200: 97.420: [ParNew: 144140K->4291K(153344K), 0.0230283 secs] 151041K->11192K(1031552K), 0.0231339 secs] [Times: user=0.01 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:01.450+0200: 108.811: [GC (Allocation Failure) 2019-06-26T09:40:01.450+0200: 108.811: [ParNew: 140611K->13047K(153344K), 0.0280423 secs] 147512K->19948K(1031552K), 0.0281406 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:02.463+0200: 109.825: [GC (Allocation Failure) 2019-06-26T09:40:02.463+0200: 109.825: [ParNew: 149367K->7162K(153344K), 0.0591644 secs] 156268K->22446K(1031552K), 0.0592586 secs] [Times: user=0.07 sys=0.00, real=0.06 secs] 
2019-06-26T09:40:03.035+0200: 110.396: [GC (Allocation Failure) 2019-06-26T09:40:03.035+0200: 110.396: [ParNew: 143482K->3056K(153344K), 0.0211872 secs] 158766K->18340K(1031552K), 0.0212620 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:05.485+0200: 112.846: [GC (Allocation Failure) 2019-06-26T09:40:05.485+0200: 112.846: [ParNew: 139376K->3838K(153344K), 0.0229737 secs] 154660K->19122K(1031552K), 0.0230716 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:06.486+0200: 113.848: [GC (Allocation Failure) 2019-06-26T09:40:06.486+0200: 113.848: [ParNew: 140158K->8095K(153344K), 0.0634246 secs] 155442K->23379K(1031552K), 0.0635210 secs] [Times: user=0.04 sys=0.00, real=0.06 secs] 
2019-06-26T09:40:06.561+0200: 113.923: [GC (CMS Initial Mark) [1 CMS-initial-mark: 15284K(878208K)] 31518K(1031552K), 0.0224020 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:06.584+0200: 113.945: [CMS-concurrent-mark-start]
2019-06-26T09:40:06.630+0200: 113.992: [CMS-concurrent-mark ...
(more)
edit retag flag offensive close merge delete

Comments

Please add the content of sdc.log

Antonin gravatar imageAntonin ( 2019-06-26 08:05:08 -0500 )edit

Please capture a thread dump. If you cannot access the UI for this, you can use one of these methods: https://community.oracle.com/blogs/ramlakshmanan/2016/10/13/how-to-take-thread-dumps-8-options <-here

jeff gravatar imagejeff ( 2019-06-26 09:19:30 -0500 )edit

Thanks for posting more info @Nightwatch. I edited it into your question, since it wasn't really an answer :-)

metadaddy gravatar imagemetadaddy ( 2019-06-26 11:29:30 -0500 )edit
1

Sorry @metadaddy! I always use in the wrong way this site... Lesson learned!

Nightwatch gravatar imageNightwatch ( 2019-06-26 11:39:28 -0500 )edit

I've updated the question. The service has finally started, but unlike the previous versions of SDC (3.8.0, 3.8.2) it took a long time to bring up the daemon." ClassLoaderStageLibraryTask - Reading from Repository Manifest URL: http://archives.streamsets.com... " sections take long time to complete

Nightwatch gravatar imageNightwatch ( 2019-06-27 03:55:22 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2019-06-28 09:28:30 -0500

Nightwatch gravatar image

I finally managed to identify the problem. The server could not reach (due to dns resolve issues) the resources listed in the sdc.log in the section:

2019-06-28 16:16:55,883 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  ClassLoaderStageLibraryTask - Reading from Repository Manifest URL: http://archives.streamsets.com/datacollector/3.9.1/tarball/repository.manifest.json
2019-06-28 16:17:23,200 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  ClassLoaderStageLibraryTask - Reading from Repository Manifest URL: http://archives.streamsets.com/datacollector/3.9.1/tarball/enterprise/repository.manifest.json
2019-06-28 16:17:23,726 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  ClassLoaderStageLibraryTask - Reading from Repository Manifest URL: http://archives.streamsets.com/datacollector/3.9.1/legacy/repository.manifest.json

So the startup procedure was waiting an indefinitely amount of time... The suggestion was taken from this post: https://www.codetd.com/en/article/641...

I apologize for the time wasted trying to solve a problem that was clearly not attributable to SDC...

edit flag offensive delete link more

Comments

No problem - and thanks for writing up the solution!

metadaddy gravatar imagemetadaddy ( 2019-07-02 22:58:47 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-06-26 02:53:44 -0500

Seen: 558 times

Last updated: Jun 28