Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Cannot start SDC 3.9.0

Hi,

I've upgraded SDC from 3.8.2 to 3.9.0 but after that the service doesn't start. I can see on gc.log that maybe something related to java memory issues.

Java HotSpot(TM) 64-Bit Server VM (25.201-b09) for linux-amd64 JRE (1.8.0_201-b09), built on Dec 15 2018 11:35:59 by "java_re" with gcc 7.3.0
Memory: 4k page, physical 8009556k(7749060k free), swap 4194300k(4194300k free)
CommandLine flags: -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/sdc/sdc_heapdump_1561534689.hprof -XX:InitialHeapSize=1073741824 -XX:MaxHeapSize=1073741824 -XX:MaxNewSize=174485504 -XX:MaxTenuringThreshold=6 -XX:NewSize=174485504 -XX:OldPLABSize=16 -XX:OldSize=348971008 -XX:-OmitStackTraceInFastThrow -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC 
2019-06-26T09:38:50.464+0200: 37.826: [GC (Allocation Failure) 2019-06-26T09:38:50.465+0200: 37.826: [ParNew: 136320K->12245K(153344K), 0.0863098 secs] 136320K->12245K(1031552K), 0.0864338 secs] [Times: user=0.03 sys=0.00, real=0.09 secs] 
2019-06-26T09:39:23.982+0200: 71.344: [GC (Allocation Failure) 2019-06-26T09:39:23.982+0200: 71.344: [ParNew: 148565K->7820K(153344K), 0.1149114 secs] 148565K->14721K(1031552K), 0.1150215 secs] [Times: user=0.07 sys=0.00, real=0.12 secs] 
2019-06-26T09:39:50.059+0200: 97.420: [GC (Allocation Failure) 2019-06-26T09:39:50.059+0200: 97.420: [ParNew: 144140K->4291K(153344K), 0.0230283 secs] 151041K->11192K(1031552K), 0.0231339 secs] [Times: user=0.01 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:01.450+0200: 108.811: [GC (Allocation Failure) 2019-06-26T09:40:01.450+0200: 108.811: [ParNew: 140611K->13047K(153344K), 0.0280423 secs] 147512K->19948K(1031552K), 0.0281406 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:02.463+0200: 109.825: [GC (Allocation Failure) 2019-06-26T09:40:02.463+0200: 109.825: [ParNew: 149367K->7162K(153344K), 0.0591644 secs] 156268K->22446K(1031552K), 0.0592586 secs] [Times: user=0.07 sys=0.00, real=0.06 secs] 
2019-06-26T09:40:03.035+0200: 110.396: [GC (Allocation Failure) 2019-06-26T09:40:03.035+0200: 110.396: [ParNew: 143482K->3056K(153344K), 0.0211872 secs] 158766K->18340K(1031552K), 0.0212620 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:05.485+0200: 112.846: [GC (Allocation Failure) 2019-06-26T09:40:05.485+0200: 112.846: [ParNew: 139376K->3838K(153344K), 0.0229737 secs] 154660K->19122K(1031552K), 0.0230716 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:06.486+0200: 113.848: [GC (Allocation Failure) 2019-06-26T09:40:06.486+0200: 113.848: [ParNew: 140158K->8095K(153344K), 0.0634246 secs] 155442K->23379K(1031552K), 0.0635210 secs] [Times: user=0.04 sys=0.00, real=0.06 secs] 
2019-06-26T09:40:06.561+0200: 113.923: [GC (CMS Initial Mark) [1 CMS-initial-mark: 15284K(878208K)] 31518K(1031552K), 0.0224020 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:06.584+0200: 113.945: [CMS-concurrent-mark-start]
2019-06-26T09:40:06.630+0200: 113.992: [CMS-concurrent-mark: 0.046/0.046 secs] [Times: user=0.08 sys=0.00, real=0.05 secs] 
2019-06-26T09:40:06.630+0200: 113.992: [CMS-concurrent-preclean-start]
2019-06-26T09:40:06.633+0200: 113.994: [CMS-concurrent-preclean: 0.003/0.003 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:06.633+0200: 113.994: [CMS-concurrent-abortable-preclean-start]
2019-06-26T09:40:07.173+0200: 114.535: [GC (Allocation Failure) 2019-06-26T09:40:07.173+0200: 114.535: [ParNew2019-06-26T09:40:07.203+0200: 114.565: [CMS-concurrent-abortable-preclean: 0.335/0.571 secs] [Times: user=0.63 sys=0.00, real=0.57 secs] 
: 144415K->8752K(153344K), 0.0378657 secs] 159699K->24036K(1031552K), 0.0379600 secs] [Times: user=0.03 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:07.243+0200: 114.605: [GC (CMS Final Remark) [YG occupancy: 16902 K (153344 K)]2019-06-26T09:40:07.244+0200: 114.605: [Rescan (parallel) , 0.0258777 secs]2019-06-26T09:40:07.269+0200: 114.631: [weak refs processing, 0.0000749 secs]2019-06-26T09:40:07.270+0200: 114.631: [class unloading, 0.0050736 secs]2019-06-26T09:40:07.275+0200: 114.636: [scrub symbol table, 0.0030306 secs]2019-06-26T09:40:07.278+0200: 114.639: [scrub string table, 0.0008257 secs][1 CMS-remark: 15284K(878208K)] 32186K(1031552K), 0.0355618 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:07.289+0200: 114.651: [CMS-concurrent-sweep-start]
2019-06-26T09:40:07.297+0200: 114.659: [CMS-concurrent-sweep: 0.008/0.008 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:07.297+0200: 114.659: [CMS-concurrent-reset-start]
2019-06-26T09:40:07.326+0200: 114.687: [CMS-concurrent-reset: 0.018/0.029 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:07.725+0200: 115.087: [GC (Allocation Failure) 2019-06-26T09:40:07.725+0200: 115.087: [ParNew: 145072K->10610K(153344K), 0.0289038 secs] 160324K->25862K(1031552K), 0.0290027 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:08.212+0200: 115.574: [GC (Allocation Failure) 2019-06-26T09:40:08.212+0200: 115.574: [ParNew: 146930K->11709K(153344K), 0.0390318 secs] 162182K->28090K(1031552K), 0.0391571 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:08.673+0200: 116.034: [GC (Allocation Failure) 2019-06-26T09:40:08.673+0200: 116.034: [ParNew: 148029K->11129K(153344K), 0.0402671 secs] 164410K->28185K(1031552K), 0.0403611 secs] [Times: user=0.03 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:09.165+0200: 116.526: [GC (Allocation Failure) 2019-06-26T09:40:09.165+0200: 116.526: [ParNew: 147449K->10823K(153344K), 0.0295319 secs] 164505K->28409K(1031552K), 0.0296430 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:09.537+0200: 116.898: [GC (Allocation Failure) 2019-06-26T09:40:09.537+0200: 116.898: [ParNew: 147143K->7257K(153344K), 0.0409611 secs] 164729K->29869K(1031552K), 0.0410538 secs] [Times: user=0.05 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:10.252+0200: 117.613: [GC (Allocation Failure) 2019-06-26T09:40:10.252+0200: 117.613: [ParNew: 143577K->6902K(153344K), 0.0487168 secs] 166189K->30034K(1031552K), 0.0488106 secs] [Times: user=0.05 sys=0.00, real=0.05 secs] 
2019-06-26T09:40:19.949+0200: 127.311: [GC (Allocation Failure) 2019-06-26T09:40:19.950+0200: 127.311: [ParNew: 143222K->11748K(153344K), 0.0436810 secs] 166354K->35323K(1031552K), 0.0438189 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:20.595+0200: 127.956: [GC (Allocation Failure) 2019-06-26T09:40:20.595+0200: 127.956: [ParNew: 148068K->12469K(153344K), 0.0436302 secs] 171643K->37068K(1031552K), 0.0437625 secs] [Times: user=0.04 sys=0.00, real=0.05 secs] 
2019-06-26T09:40:20.991+0200: 128.353: [GC (Allocation Failure) 2019-06-26T09:40:20.991+0200: 128.353: [ParNew: 148789K->9785K(153344K), 0.0323473 secs] 173388K->34384K(1031552K), 0.0324717 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:21.272+0200: 128.633: [GC (Allocation Failure) 2019-06-26T09:40:21.272+0200: 128.633: [ParNew: 146105K->9338K(153344K), 0.0200447 secs] 170704K->34602K(1031552K), 0.0201699 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:21.707+0200: 129.068: [GC (Allocation Failure) 2019-06-26T09:40:21.707+0200: 129.068: [ParNew: 145658K->10012K(153344K), 0.0286655 secs] 170922K->35720K(1031552K), 0.0287987 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:23.251+0200: 130.613: [GC (Allocation Failure) 2019-06-26T09:40:23.251+0200: 130.613: [ParNew: 146332K->12312K(153344K), 0.0270685 secs] 172040K->39280K(1031552K), 0.0272055 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:25.320+0200: 132.682: [GC (Allocation Failure) 2019-06-26T09:40:25.320+0200: 132.682: [ParNew: 148632K->12318K(153344K), 0.0357669 secs] 175600K->42343K(1031552K), 0.0358996 secs] [Times: user=0.07 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:25.568+0200: 132.930: [GC (Allocation Failure) 2019-06-26T09:40:25.568+0200: 132.930: [ParNew: 148638K->7849K(153344K), 0.0143115 secs] 178663K->38629K(1031552K), 0.0144382 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:31.386+0200: 138.748: [GC (Allocation Failure) 2019-06-26T09:40:31.386+0200: 138.748: [ParNew: 144163K->12286K(153344K), 0.0169158 secs] 174943K->43767K(1031552K), 0.0170368 secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:31.685+0200: 139.047: [GC (Allocation Failure) 2019-06-26T09:40:31.686+0200: 139.047: [ParNew: 148606K->9860K(153344K), 0.0301863 secs] 180087K->41830K(1031552K), 0.0303219 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:31.923+0200: 139.284: [GC (Allocation Failure) 2019-06-26T09:40:31.923+0200: 139.284: [ParNew: 146180K->10177K(153344K), 0.0300010 secs] 178150K->42770K(1031552K), 0.0301394 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:32.166+0200: 139.528: [GC (Allocation Failure) 2019-06-26T09:40:32.167+0200: 139.528: [ParNew: 146497K->10337K(153344K), 0.0187723 secs] 179090K->44185K(1031552K), 0.0188992 secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:32.480+0200: 139.842: [GC (Allocation Failure) 2019-06-26T09:40:32.480+0200: 139.842: [ParNew: 146657K->10576K(153344K), 0.0175061 secs] 180505K->46028K(1031552K), 0.0176373 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:35.179+0200: 142.540: [GC (Allocation Failure) 2019-06-26T09:40:35.179+0200: 142.540: [ParNew: 146896K->13689K(153344K), 0.0409025 secs] 182348K->49981K(1031552K), 0.0410205 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:35.459+0200: 142.820: [GC (Allocation Failure) 2019-06-26T09:40:35.459+0200: 142.820: [ParNew: 150009K->12614K(153344K), 0.0199374 secs] 186301K->51418K(1031552K), 0.0200658 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:38.059+0200: 145.421: [GC (Allocation Failure) 2019-06-26T09:40:38.059+0200: 145.421: [ParNew: 148934K->13399K(153344K), 0.0210353 secs] 187738K->52203K(1031552K), 0.0211810 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:43.064+0200: 150.426: [GC (Allocation Failure) 2019-06-26T09:40:43.064+0200: 150.426: [ParNew: 149719K->13295K(153344K), 0.0249492 secs] 188523K->54550K(1031552K), 0.0250791 secs] [Times: user=0.05 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:43.340+0200: 150.702: [GC (Allocation Failure) 2019-06-26T09:40:43.341+0200: 150.702: [ParNew: 149615K->8115K(153344K), 0.0202175 secs] 190870K->53376K(1031552K), 0.0203387 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:43.366+0200: 150.727: [GC (CMS Initial Mark) [1 CMS-initial-mark: 45261K(878208K)] 58813K(1031552K), 0.0068261 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:43.373+0200: 150.734: [CMS-concurrent-mark-start]
2019-06-26T09:40:43.450+0200: 150.812: [CMS-concurrent-mark: 0.078/0.078 secs] [Times: user=0.16 sys=0.00, real=0.08 secs] 
2019-06-26T09:40:43.450+0200: 150.812: [CMS-concurrent-preclean-start]
2019-06-26T09:40:43.453+0200: 150.815: [CMS-concurrent-preclean: 0.003/0.003 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:43.453+0200: 150.815: [CMS-concurrent-abortable-preclean-start]
2019-06-26T09:40:43.585+0200: 150.946: [GC (Allocation Failure) 2019-06-26T09:40:43.585+0200: 150.946: [ParNew: 144435K->7439K(153344K), 0.0119117 secs] 189696K->52701K(1031552K), 0.0120142 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:44.402+0200: 151.764: [GC (Allocation Failure) 2019-06-26T09:40:44.402+0200: 151.764: [ParNew: 143759K->10470K(153344K), 0.0138868 secs] 189021K->55731K(1031552K), 0.0140334 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:44.417+0200: 151.778: [CMS-concurrent-abortable-preclean: 0.396/0.963 secs] [Times: user=1.22 sys=0.00, real=0.97 secs] 
2019-06-26T09:40:44.418+0200: 151.779: [GC (CMS Final Remark) [YG occupancy: 13192 K (153344 K)]2019-06-26T09:40:44.418+0200: 151.779: [Rescan (parallel) , 0.0062470 secs]2019-06-26T09:40:44.424+0200: 151.785: [weak refs processing, 0.0000659 secs]2019-06-26T09:40:44.424+0200: 151.785: [class unloading, 0.0078585 secs]2019-06-26T09:40:44.432+0200: 151.793: [scrub symbol table, 0.0040422 secs]2019-06-26T09:40:44.436+0200: 151.797: [scrub string table, 0.0010877 secs][1 CMS-remark: 45261K(878208K)] 58454K(1031552K), 0.0197323 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:44.438+0200: 151.799: [CMS-concurrent-sweep-start]
2019-06-26T09:40:44.469+0200: 151.831: [CMS-concurrent-sweep: 0.032/0.032 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:44.469+0200: 151.831: [CMS-concurrent-reset-start]
2019-06-26T09:40:44.473+0200: 151.834: [CMS-concurrent-reset: 0.003/0.003 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:47.980+0200: 155.341: [GC (Allocation Failure) 2019-06-26T09:40:47.980+0200: 155.341: [ParNew: 146790K->10524K(153344K), 0.0185544 secs] 191767K->56403K(1031552K), 0.0187028 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:48.232+0200: 155.594: [GC (Allocation Failure) 2019-06-26T09:40:48.232+0200: 155.594: [ParNew: 146844K->7002K(153344K), 0.0159609 secs] 192723K->54194K(1031552K), 0.0160909 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:48.515+0200: 155.876: [GC (Allocation Failure) 2019-06-26T09:40:48.515+0200: 155.876: [ParNew: 143322K->10230K(153344K), 0.0166830 secs] 190514K->59060K(1031552K), 0.0168019 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:48.812+0200: 156.173: [GC (Allocation Failure) 2019-06-26T09:40:48.812+0200: 156.173: [ParNew: 146550K->8420K(153344K), 0.0203064 secs] 195380K->57741K(1031552K), 0.0204353 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:49.074+0200: 156.436: [GC (Allocation Failure) 2019-06-26T09:40:49.074+0200: 156.436: [ParNew: 144740K->8350K(153344K), 0.0188808 secs] 194061K->58300K(1031552K), 0.0190149 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:53.186+0200: 160.547: [GC (Allocation Failure) 2019-06-26T09:40:53.186+0200: 160.547: [ParNew: 144670K->9929K(153344K), 0.0178093 secs] 194620K->61153K(1031552K), 0.0179394 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:55.021+0200: 162.382: [GC (Allocation Failure) 2019-06-26T09:40:55.021+0200: 162.382: [ParNew: 146249K->10023K(153344K), 0.0342168 secs] 197473K->62884K(1031552K), 0.0343433 secs] [Times: user=0.07 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:58.877+0200: 166.239: [GC (Allocation Failure) 2019-06-26T09:40:58.877+0200: 166.239: [ParNew: 146343K->10757K(153344K), 0.0266899 secs] 199204K->64189K(1031552K), 0.0268154 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:59.216+0200: 166.577: [GC (Allocation Failure) 2019-06-26T09:40:59.216+0200: 166.577: [ParNew: 147077K->10578K(153344K), 0.0337889 secs] 200509K->64983K(1031552K), 0.0339147 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:59.475+0200: 166.836: [GC (Allocation Failure) 2019-06-26T09:40:59.475+0200: 166.836: [ParNew: 146898K->10449K(153344K), 0.0308185 secs] 201303K->65591K(1031552K), 0.0309496 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:59.855+0200: 167.216: [GC (Allocation Failure) 2019-06-26T09:40:59.855+0200: 167.216: [ParNew: 146769K->11019K(153344K), 0.0342162 secs] 201911K->67110K(1031552K), 0.0343465 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:41:00.115+0200: 167.477: [GC (Allocation Failure) 2019-06-26T09:41:00.115+0200: 167.477: [ParNew: 147336K->10448K(153344K), 0.0172150 secs] 203426K->67660K(1031552K), 0.0173502 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:01.984+0200: 169.346: [GC (Allocation Failure) 2019-06-26T09:41:01.985+0200: 169.346: [ParNew: 146768K->11447K(153344K), 0.0222017 secs] 203980K->70213K(1031552K), 0.0223227 secs] [Times: user=0.05 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:02.214+0200: 169.575: [GC (Allocation Failure) 2019-06-26T09:41:02.214+0200: 169.575: [ParNew: 147767K->6925K(153344K), 0.0172119 secs] 206533K->67457K(1031552K), 0.0173520 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:02.427+0200: 169.788: [GC (Allocation Failure) 2019-06-26T09:41:02.427+0200: 169.788: [ParNew: 143245K->5845K(153344K), 0.0141443 secs] 203777K->66988K(1031552K), 0.0142804 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:03.446+0200: 170.807: [GC (Allocation Failure) 2019-06-26T09:41:03.446+0200: 170.807: [ParNew: 142165K->11712K(153344K), 0.0150314 secs] 203308K->73305K(1031552K), 0.0151429 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:08.352+0200: 175.714: [GC (Allocation Failure) 2019-06-26T09:41:08.352+0200: 175.714: [ParNew: 148032K->10805K(153344K), 0.0191556 secs] 209625K->73712K(1031552K), 0.0193803 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:08.627+0200: 175.989: [GC (Allocation Failure) 2019-06-26T09:41:08.627+0200: 175.989: [ParNew: 147125K->11424K(153344K), 0.0155996 secs] 210032K->75312K(1031552K), 0.0157533 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:08.851+0200: 176.213: [GC (Allocation Failure) 2019-06-26T09:41:08.851+0200: 176.213: [ParNew: 147744K->7348K(153344K), 0.0159758 secs] 211632K->72884K(1031552K), 0.0161171 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:09.083+0200: 176.445: [GC (Allocation Failure) 2019-06-26T09:41:09.083+0200: 176.445: [ParNew: 143668K->6797K(153344K), 0.0125130 secs] 209204K->72782K(1031552K), 0.0126500 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:09.471+0200: 176.833: [GC (Allocation Failure) 2019-06-26T09:41:09.471+0200: 176.833: [ParNew: 143117K->7801K(153344K), 0.0134573 secs] 209102K->74208K(1031552K), 0.0136030 secs] [Times: user=0.02 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:09.685+0200: 177.046: [GC (Allocation Failure) 2019-06-26T09:41:09.685+0200: 177.046: [ParNew: 144121K->7923K(153344K), 0.0140758 secs] 210528K->75147K(1031552K), 0.0142106 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:09.905+0200: 177.266: [GC (Allocation Failure) 2019-06-26T09:41:09.905+0200: 177.266: [ParNew: 144243K->7461K(153344K), 0.0185685 secs] 211467K->76528K(1031552K), 0.0187047 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:11.066+0200: 178.427: [GC (Allocation Failure) 2019-06-26T09:41:11.066+0200: 178.427: [ParNew: 143781K->5783K(153344K), 0.0156430 secs] 212848K->75663K(1031552K), 0.0157626 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:17.091+0200: 184.452: [GC (Allocation Failure) 2019-06-26T09:41:17.091+0200: 184.452: [ParNew: 142103K->8812K(153344K), 0.0213777 secs] 211983K->79053K(1031552K), 0.0215127 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:17.402+0200: 184.764: [GC (Allocation Failure) 2019-06-26T09:41:17.402+0200: 184.764: [ParNew: 145132K->10819K(153344K), 0.0148629 secs] 215373K->81788K(1031552K), 0.0149911 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:17.715+0200: 185.077: [GC (Allocation Failure) 2019-06-26T09:41:17.715+0200: 185.077: [ParNew: 147139K->8484K(153344K), 0.0174850 secs] 218108K->80236K(1031552K), 0.0176207 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:18.035+0200: 185.396: [GC (Allocation Failure) 2019-06-26T09:41:18.035+0200: 185.396: [ParNew: 144804K->12105K(153344K), 0.0186439 secs] 216556K->84164K(1031552K), 0.0187788 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:18.301+0200: 185.662: [GC (Allocation Failure) 2019-06-26T09:41:18.301+0200: 185.662: [ParNew: 148425K->12995K(153344K), 0.0218518 secs] 220484K->86356K(1031552K), 0.0219875 secs] [Times: user=0.05 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:18.634+0200: 185.996: [GC (Allocation Failure) 2019-06-26T09:41:18.634+0200: 185.996: [ParNew: 149315K->14491K(153344K), 0.0283266 secs] 222676K->90233K(1031552K), 0.0284645 secs] [Times: user=0.05 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:18.933+0200: 186.295: [GC (Allocation Failure) 2019-06-26T09:41:18.933+0200: 186.295: [ParNew: 150811K->15713K(153344K), 0.0303841 secs] 226553K->94447K(1031552K), 0.0305207 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:19.244+0200: 186.605: [GC (Allocation Failure) 2019-06-26T09:41:19.244+0200: 186.605: [ParNew: 152033K->14779K(153344K), 0.0217431 secs] 230767K->96395K(1031552K), 0.0218754 secs] [Times: user=0.04 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:19.555+0200: 186.917: [GC (Allocation Failure) 2019-06-26T09:41:19.556+0200: 186.917: [ParNew: 151099K->13846K(153344K), 0.0209353 secs] 232715K->97223K(1031552K), 0.0210718 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:19.848+0200: 187.210: [GC (Allocation Failure) 2019-06-26T09:41:19.848+0200: 187.210: [ParNew: 150166K->15004K(153344K), 0.0223914 secs] 233543K->101502K(1031552K), 0.0225120 secs] [Times: user=0.05 sys=0.00, real=0.02 secs]

Here the server conf: OS: Centos 7 64bit RAM: 8GB limits: ulimit -n 33000

enter code here

Cannot start SDC 3.9.0

Hi,

I've upgraded SDC from 3.8.2 to 3.9.0 but after that the service doesn't start. I can see on gc.log that maybe something related to java memory issues.

Java HotSpot(TM) 64-Bit Server VM (25.201-b09) for linux-amd64 JRE (1.8.0_201-b09), built on Dec 15 2018 11:35:59 by "java_re" with gcc 7.3.0
Memory: 4k page, physical 8009556k(7749060k free), swap 4194300k(4194300k free)
CommandLine flags: -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/sdc/sdc_heapdump_1561534689.hprof -XX:InitialHeapSize=1073741824 -XX:MaxHeapSize=1073741824 -XX:MaxNewSize=174485504 -XX:MaxTenuringThreshold=6 -XX:NewSize=174485504 -XX:OldPLABSize=16 -XX:OldSize=348971008 -XX:-OmitStackTraceInFastThrow -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC 
2019-06-26T09:38:50.464+0200: 37.826: [GC (Allocation Failure) 2019-06-26T09:38:50.465+0200: 37.826: [ParNew: 136320K->12245K(153344K), 0.0863098 secs] 136320K->12245K(1031552K), 0.0864338 secs] [Times: user=0.03 sys=0.00, real=0.09 secs] 
2019-06-26T09:39:23.982+0200: 71.344: [GC (Allocation Failure) 2019-06-26T09:39:23.982+0200: 71.344: [ParNew: 148565K->7820K(153344K), 0.1149114 secs] 148565K->14721K(1031552K), 0.1150215 secs] [Times: user=0.07 sys=0.00, real=0.12 secs] 
2019-06-26T09:39:50.059+0200: 97.420: [GC (Allocation Failure) 2019-06-26T09:39:50.059+0200: 97.420: [ParNew: 144140K->4291K(153344K), 0.0230283 secs] 151041K->11192K(1031552K), 0.0231339 secs] [Times: user=0.01 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:01.450+0200: 108.811: [GC (Allocation Failure) 2019-06-26T09:40:01.450+0200: 108.811: [ParNew: 140611K->13047K(153344K), 0.0280423 secs] 147512K->19948K(1031552K), 0.0281406 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:02.463+0200: 109.825: [GC (Allocation Failure) 2019-06-26T09:40:02.463+0200: 109.825: [ParNew: 149367K->7162K(153344K), 0.0591644 secs] 156268K->22446K(1031552K), 0.0592586 secs] [Times: user=0.07 sys=0.00, real=0.06 secs] 
2019-06-26T09:40:03.035+0200: 110.396: [GC (Allocation Failure) 2019-06-26T09:40:03.035+0200: 110.396: [ParNew: 143482K->3056K(153344K), 0.0211872 secs] 158766K->18340K(1031552K), 0.0212620 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:05.485+0200: 112.846: [GC (Allocation Failure) 2019-06-26T09:40:05.485+0200: 112.846: [ParNew: 139376K->3838K(153344K), 0.0229737 secs] 154660K->19122K(1031552K), 0.0230716 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:06.486+0200: 113.848: [GC (Allocation Failure) 2019-06-26T09:40:06.486+0200: 113.848: [ParNew: 140158K->8095K(153344K), 0.0634246 secs] 155442K->23379K(1031552K), 0.0635210 secs] [Times: user=0.04 sys=0.00, real=0.06 secs] 
2019-06-26T09:40:06.561+0200: 113.923: [GC (CMS Initial Mark) [1 CMS-initial-mark: 15284K(878208K)] 31518K(1031552K), 0.0224020 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:06.584+0200: 113.945: [CMS-concurrent-mark-start]
2019-06-26T09:40:06.630+0200: 113.992: [CMS-concurrent-mark: 0.046/0.046 secs] [Times: user=0.08 sys=0.00, real=0.05 secs] 
2019-06-26T09:40:06.630+0200: 113.992: [CMS-concurrent-preclean-start]
2019-06-26T09:40:06.633+0200: 113.994: [CMS-concurrent-preclean: 0.003/0.003 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:06.633+0200: 113.994: [CMS-concurrent-abortable-preclean-start]
2019-06-26T09:40:07.173+0200: 114.535: [GC (Allocation Failure) 2019-06-26T09:40:07.173+0200: 114.535: [ParNew2019-06-26T09:40:07.203+0200: 114.565: [CMS-concurrent-abortable-preclean: 0.335/0.571 secs] [Times: user=0.63 sys=0.00, real=0.57 secs] 
: 144415K->8752K(153344K), 0.0378657 secs] 159699K->24036K(1031552K), 0.0379600 secs] [Times: user=0.03 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:07.243+0200: 114.605: [GC (CMS Final Remark) [YG occupancy: 16902 K (153344 K)]2019-06-26T09:40:07.244+0200: 114.605: [Rescan (parallel) , 0.0258777 secs]2019-06-26T09:40:07.269+0200: 114.631: [weak refs processing, 0.0000749 secs]2019-06-26T09:40:07.270+0200: 114.631: [class unloading, 0.0050736 secs]2019-06-26T09:40:07.275+0200: 114.636: [scrub symbol table, 0.0030306 secs]2019-06-26T09:40:07.278+0200: 114.639: [scrub string table, 0.0008257 secs][1 CMS-remark: 15284K(878208K)] 32186K(1031552K), 0.0355618 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:07.289+0200: 114.651: [CMS-concurrent-sweep-start]
2019-06-26T09:40:07.297+0200: 114.659: [CMS-concurrent-sweep: 0.008/0.008 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:07.297+0200: 114.659: [CMS-concurrent-reset-start]
2019-06-26T09:40:07.326+0200: 114.687: [CMS-concurrent-reset: 0.018/0.029 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:07.725+0200: 115.087: [GC (Allocation Failure) 2019-06-26T09:40:07.725+0200: 115.087: [ParNew: 145072K->10610K(153344K), 0.0289038 secs] 160324K->25862K(1031552K), 0.0290027 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:08.212+0200: 115.574: [GC (Allocation Failure) 2019-06-26T09:40:08.212+0200: 115.574: [ParNew: 146930K->11709K(153344K), 0.0390318 secs] 162182K->28090K(1031552K), 0.0391571 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:08.673+0200: 116.034: [GC (Allocation Failure) 2019-06-26T09:40:08.673+0200: 116.034: [ParNew: 148029K->11129K(153344K), 0.0402671 secs] 164410K->28185K(1031552K), 0.0403611 secs] [Times: user=0.03 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:09.165+0200: 116.526: [GC (Allocation Failure) 2019-06-26T09:40:09.165+0200: 116.526: [ParNew: 147449K->10823K(153344K), 0.0295319 secs] 164505K->28409K(1031552K), 0.0296430 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:09.537+0200: 116.898: [GC (Allocation Failure) 2019-06-26T09:40:09.537+0200: 116.898: [ParNew: 147143K->7257K(153344K), 0.0409611 secs] 164729K->29869K(1031552K), 0.0410538 secs] [Times: user=0.05 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:10.252+0200: 117.613: [GC (Allocation Failure) 2019-06-26T09:40:10.252+0200: 117.613: [ParNew: 143577K->6902K(153344K), 0.0487168 secs] 166189K->30034K(1031552K), 0.0488106 secs] [Times: user=0.05 sys=0.00, real=0.05 secs] 
2019-06-26T09:40:19.949+0200: 127.311: [GC (Allocation Failure) 2019-06-26T09:40:19.950+0200: 127.311: [ParNew: 143222K->11748K(153344K), 0.0436810 secs] 166354K->35323K(1031552K), 0.0438189 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:20.595+0200: 127.956: [GC (Allocation Failure) 2019-06-26T09:40:20.595+0200: 127.956: [ParNew: 148068K->12469K(153344K), 0.0436302 secs] 171643K->37068K(1031552K), 0.0437625 secs] [Times: user=0.04 sys=0.00, real=0.05 secs] 
2019-06-26T09:40:20.991+0200: 128.353: [GC (Allocation Failure) 2019-06-26T09:40:20.991+0200: 128.353: [ParNew: 148789K->9785K(153344K), 0.0323473 secs] 173388K->34384K(1031552K), 0.0324717 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:21.272+0200: 128.633: [GC (Allocation Failure) 2019-06-26T09:40:21.272+0200: 128.633: [ParNew: 146105K->9338K(153344K), 0.0200447 secs] 170704K->34602K(1031552K), 0.0201699 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:21.707+0200: 129.068: [GC (Allocation Failure) 2019-06-26T09:40:21.707+0200: 129.068: [ParNew: 145658K->10012K(153344K), 0.0286655 secs] 170922K->35720K(1031552K), 0.0287987 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:23.251+0200: 130.613: [GC (Allocation Failure) 2019-06-26T09:40:23.251+0200: 130.613: [ParNew: 146332K->12312K(153344K), 0.0270685 secs] 172040K->39280K(1031552K), 0.0272055 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:25.320+0200: 132.682: [GC (Allocation Failure) 2019-06-26T09:40:25.320+0200: 132.682: [ParNew: 148632K->12318K(153344K), 0.0357669 secs] 175600K->42343K(1031552K), 0.0358996 secs] [Times: user=0.07 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:25.568+0200: 132.930: [GC (Allocation Failure) 2019-06-26T09:40:25.568+0200: 132.930: [ParNew: 148638K->7849K(153344K), 0.0143115 secs] 178663K->38629K(1031552K), 0.0144382 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:31.386+0200: 138.748: [GC (Allocation Failure) 2019-06-26T09:40:31.386+0200: 138.748: [ParNew: 144163K->12286K(153344K), 0.0169158 secs] 174943K->43767K(1031552K), 0.0170368 secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:31.685+0200: 139.047: [GC (Allocation Failure) 2019-06-26T09:40:31.686+0200: 139.047: [ParNew: 148606K->9860K(153344K), 0.0301863 secs] 180087K->41830K(1031552K), 0.0303219 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:31.923+0200: 139.284: [GC (Allocation Failure) 2019-06-26T09:40:31.923+0200: 139.284: [ParNew: 146180K->10177K(153344K), 0.0300010 secs] 178150K->42770K(1031552K), 0.0301394 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:32.166+0200: 139.528: [GC (Allocation Failure) 2019-06-26T09:40:32.167+0200: 139.528: [ParNew: 146497K->10337K(153344K), 0.0187723 secs] 179090K->44185K(1031552K), 0.0188992 secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:32.480+0200: 139.842: [GC (Allocation Failure) 2019-06-26T09:40:32.480+0200: 139.842: [ParNew: 146657K->10576K(153344K), 0.0175061 secs] 180505K->46028K(1031552K), 0.0176373 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:35.179+0200: 142.540: [GC (Allocation Failure) 2019-06-26T09:40:35.179+0200: 142.540: [ParNew: 146896K->13689K(153344K), 0.0409025 secs] 182348K->49981K(1031552K), 0.0410205 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:35.459+0200: 142.820: [GC (Allocation Failure) 2019-06-26T09:40:35.459+0200: 142.820: [ParNew: 150009K->12614K(153344K), 0.0199374 secs] 186301K->51418K(1031552K), 0.0200658 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:38.059+0200: 145.421: [GC (Allocation Failure) 2019-06-26T09:40:38.059+0200: 145.421: [ParNew: 148934K->13399K(153344K), 0.0210353 secs] 187738K->52203K(1031552K), 0.0211810 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:43.064+0200: 150.426: [GC (Allocation Failure) 2019-06-26T09:40:43.064+0200: 150.426: [ParNew: 149719K->13295K(153344K), 0.0249492 secs] 188523K->54550K(1031552K), 0.0250791 secs] [Times: user=0.05 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:43.340+0200: 150.702: [GC (Allocation Failure) 2019-06-26T09:40:43.341+0200: 150.702: [ParNew: 149615K->8115K(153344K), 0.0202175 secs] 190870K->53376K(1031552K), 0.0203387 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:43.366+0200: 150.727: [GC (CMS Initial Mark) [1 CMS-initial-mark: 45261K(878208K)] 58813K(1031552K), 0.0068261 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:43.373+0200: 150.734: [CMS-concurrent-mark-start]
2019-06-26T09:40:43.450+0200: 150.812: [CMS-concurrent-mark: 0.078/0.078 secs] [Times: user=0.16 sys=0.00, real=0.08 secs] 
2019-06-26T09:40:43.450+0200: 150.812: [CMS-concurrent-preclean-start]
2019-06-26T09:40:43.453+0200: 150.815: [CMS-concurrent-preclean: 0.003/0.003 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:43.453+0200: 150.815: [CMS-concurrent-abortable-preclean-start]
2019-06-26T09:40:43.585+0200: 150.946: [GC (Allocation Failure) 2019-06-26T09:40:43.585+0200: 150.946: [ParNew: 144435K->7439K(153344K), 0.0119117 secs] 189696K->52701K(1031552K), 0.0120142 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:44.402+0200: 151.764: [GC (Allocation Failure) 2019-06-26T09:40:44.402+0200: 151.764: [ParNew: 143759K->10470K(153344K), 0.0138868 secs] 189021K->55731K(1031552K), 0.0140334 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:44.417+0200: 151.778: [CMS-concurrent-abortable-preclean: 0.396/0.963 secs] [Times: user=1.22 sys=0.00, real=0.97 secs] 
2019-06-26T09:40:44.418+0200: 151.779: [GC (CMS Final Remark) [YG occupancy: 13192 K (153344 K)]2019-06-26T09:40:44.418+0200: 151.779: [Rescan (parallel) , 0.0062470 secs]2019-06-26T09:40:44.424+0200: 151.785: [weak refs processing, 0.0000659 secs]2019-06-26T09:40:44.424+0200: 151.785: [class unloading, 0.0078585 secs]2019-06-26T09:40:44.432+0200: 151.793: [scrub symbol table, 0.0040422 secs]2019-06-26T09:40:44.436+0200: 151.797: [scrub string table, 0.0010877 secs][1 CMS-remark: 45261K(878208K)] 58454K(1031552K), 0.0197323 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:44.438+0200: 151.799: [CMS-concurrent-sweep-start]
2019-06-26T09:40:44.469+0200: 151.831: [CMS-concurrent-sweep: 0.032/0.032 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:44.469+0200: 151.831: [CMS-concurrent-reset-start]
2019-06-26T09:40:44.473+0200: 151.834: [CMS-concurrent-reset: 0.003/0.003 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:47.980+0200: 155.341: [GC (Allocation Failure) 2019-06-26T09:40:47.980+0200: 155.341: [ParNew: 146790K->10524K(153344K), 0.0185544 secs] 191767K->56403K(1031552K), 0.0187028 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:48.232+0200: 155.594: [GC (Allocation Failure) 2019-06-26T09:40:48.232+0200: 155.594: [ParNew: 146844K->7002K(153344K), 0.0159609 secs] 192723K->54194K(1031552K), 0.0160909 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:48.515+0200: 155.876: [GC (Allocation Failure) 2019-06-26T09:40:48.515+0200: 155.876: [ParNew: 143322K->10230K(153344K), 0.0166830 secs] 190514K->59060K(1031552K), 0.0168019 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:48.812+0200: 156.173: [GC (Allocation Failure) 2019-06-26T09:40:48.812+0200: 156.173: [ParNew: 146550K->8420K(153344K), 0.0203064 secs] 195380K->57741K(1031552K), 0.0204353 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:49.074+0200: 156.436: [GC (Allocation Failure) 2019-06-26T09:40:49.074+0200: 156.436: [ParNew: 144740K->8350K(153344K), 0.0188808 secs] 194061K->58300K(1031552K), 0.0190149 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:53.186+0200: 160.547: [GC (Allocation Failure) 2019-06-26T09:40:53.186+0200: 160.547: [ParNew: 144670K->9929K(153344K), 0.0178093 secs] 194620K->61153K(1031552K), 0.0179394 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:55.021+0200: 162.382: [GC (Allocation Failure) 2019-06-26T09:40:55.021+0200: 162.382: [ParNew: 146249K->10023K(153344K), 0.0342168 secs] 197473K->62884K(1031552K), 0.0343433 secs] [Times: user=0.07 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:58.877+0200: 166.239: [GC (Allocation Failure) 2019-06-26T09:40:58.877+0200: 166.239: [ParNew: 146343K->10757K(153344K), 0.0266899 secs] 199204K->64189K(1031552K), 0.0268154 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:59.216+0200: 166.577: [GC (Allocation Failure) 2019-06-26T09:40:59.216+0200: 166.577: [ParNew: 147077K->10578K(153344K), 0.0337889 secs] 200509K->64983K(1031552K), 0.0339147 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:59.475+0200: 166.836: [GC (Allocation Failure) 2019-06-26T09:40:59.475+0200: 166.836: [ParNew: 146898K->10449K(153344K), 0.0308185 secs] 201303K->65591K(1031552K), 0.0309496 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:59.855+0200: 167.216: [GC (Allocation Failure) 2019-06-26T09:40:59.855+0200: 167.216: [ParNew: 146769K->11019K(153344K), 0.0342162 secs] 201911K->67110K(1031552K), 0.0343465 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:41:00.115+0200: 167.477: [GC (Allocation Failure) 2019-06-26T09:41:00.115+0200: 167.477: [ParNew: 147336K->10448K(153344K), 0.0172150 secs] 203426K->67660K(1031552K), 0.0173502 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:01.984+0200: 169.346: [GC (Allocation Failure) 2019-06-26T09:41:01.985+0200: 169.346: [ParNew: 146768K->11447K(153344K), 0.0222017 secs] 203980K->70213K(1031552K), 0.0223227 secs] [Times: user=0.05 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:02.214+0200: 169.575: [GC (Allocation Failure) 2019-06-26T09:41:02.214+0200: 169.575: [ParNew: 147767K->6925K(153344K), 0.0172119 secs] 206533K->67457K(1031552K), 0.0173520 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:02.427+0200: 169.788: [GC (Allocation Failure) 2019-06-26T09:41:02.427+0200: 169.788: [ParNew: 143245K->5845K(153344K), 0.0141443 secs] 203777K->66988K(1031552K), 0.0142804 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:03.446+0200: 170.807: [GC (Allocation Failure) 2019-06-26T09:41:03.446+0200: 170.807: [ParNew: 142165K->11712K(153344K), 0.0150314 secs] 203308K->73305K(1031552K), 0.0151429 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:08.352+0200: 175.714: [GC (Allocation Failure) 2019-06-26T09:41:08.352+0200: 175.714: [ParNew: 148032K->10805K(153344K), 0.0191556 secs] 209625K->73712K(1031552K), 0.0193803 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:08.627+0200: 175.989: [GC (Allocation Failure) 2019-06-26T09:41:08.627+0200: 175.989: [ParNew: 147125K->11424K(153344K), 0.0155996 secs] 210032K->75312K(1031552K), 0.0157533 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:08.851+0200: 176.213: [GC (Allocation Failure) 2019-06-26T09:41:08.851+0200: 176.213: [ParNew: 147744K->7348K(153344K), 0.0159758 secs] 211632K->72884K(1031552K), 0.0161171 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:09.083+0200: 176.445: [GC (Allocation Failure) 2019-06-26T09:41:09.083+0200: 176.445: [ParNew: 143668K->6797K(153344K), 0.0125130 secs] 209204K->72782K(1031552K), 0.0126500 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:09.471+0200: 176.833: [GC (Allocation Failure) 2019-06-26T09:41:09.471+0200: 176.833: [ParNew: 143117K->7801K(153344K), 0.0134573 secs] 209102K->74208K(1031552K), 0.0136030 secs] [Times: user=0.02 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:09.685+0200: 177.046: [GC (Allocation Failure) 2019-06-26T09:41:09.685+0200: 177.046: [ParNew: 144121K->7923K(153344K), 0.0140758 secs] 210528K->75147K(1031552K), 0.0142106 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:09.905+0200: 177.266: [GC (Allocation Failure) 2019-06-26T09:41:09.905+0200: 177.266: [ParNew: 144243K->7461K(153344K), 0.0185685 secs] 211467K->76528K(1031552K), 0.0187047 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:11.066+0200: 178.427: [GC (Allocation Failure) 2019-06-26T09:41:11.066+0200: 178.427: [ParNew: 143781K->5783K(153344K), 0.0156430 secs] 212848K->75663K(1031552K), 0.0157626 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:17.091+0200: 184.452: [GC (Allocation Failure) 2019-06-26T09:41:17.091+0200: 184.452: [ParNew: 142103K->8812K(153344K), 0.0213777 secs] 211983K->79053K(1031552K), 0.0215127 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:17.402+0200: 184.764: [GC (Allocation Failure) 2019-06-26T09:41:17.402+0200: 184.764: [ParNew: 145132K->10819K(153344K), 0.0148629 secs] 215373K->81788K(1031552K), 0.0149911 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:17.715+0200: 185.077: [GC (Allocation Failure) 2019-06-26T09:41:17.715+0200: 185.077: [ParNew: 147139K->8484K(153344K), 0.0174850 secs] 218108K->80236K(1031552K), 0.0176207 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:18.035+0200: 185.396: [GC (Allocation Failure) 2019-06-26T09:41:18.035+0200: 185.396: [ParNew: 144804K->12105K(153344K), 0.0186439 secs] 216556K->84164K(1031552K), 0.0187788 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:18.301+0200: 185.662: [GC (Allocation Failure) 2019-06-26T09:41:18.301+0200: 185.662: [ParNew: 148425K->12995K(153344K), 0.0218518 secs] 220484K->86356K(1031552K), 0.0219875 secs] [Times: user=0.05 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:18.634+0200: 185.996: [GC (Allocation Failure) 2019-06-26T09:41:18.634+0200: 185.996: [ParNew: 149315K->14491K(153344K), 0.0283266 secs] 222676K->90233K(1031552K), 0.0284645 secs] [Times: user=0.05 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:18.933+0200: 186.295: [GC (Allocation Failure) 2019-06-26T09:41:18.933+0200: 186.295: [ParNew: 150811K->15713K(153344K), 0.0303841 secs] 226553K->94447K(1031552K), 0.0305207 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:19.244+0200: 186.605: [GC (Allocation Failure) 2019-06-26T09:41:19.244+0200: 186.605: [ParNew: 152033K->14779K(153344K), 0.0217431 secs] 230767K->96395K(1031552K), 0.0218754 secs] [Times: user=0.04 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:19.555+0200: 186.917: [GC (Allocation Failure) 2019-06-26T09:41:19.556+0200: 186.917: [ParNew: 151099K->13846K(153344K), 0.0209353 secs] 232715K->97223K(1031552K), 0.0210718 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:19.848+0200: 187.210: [GC (Allocation Failure) 2019-06-26T09:41:19.848+0200: 187.210: [ParNew: 150166K->15004K(153344K), 0.0223914 secs] 233543K->101502K(1031552K), 0.0225120 secs] [Times: user=0.05 sys=0.00, real=0.02 secs]

Here the server conf: OS: Centos 7 64bit RAM: 8GB limits: ulimit -n 33000

enter code hereHere the thread dump, captured with command sudo -u sdc jstack -l <pid> > /tmp/threadDump.txt

2019-06-26 18:01:49

Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.201-b09 mixed mode):

"Attach Listener" #107 daemon prio=9 os_prio=0 tid=0x00007f6ae8001000 nid=0x55db waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"FileWatchdog" #10 daemon prio=5 os_prio=0 tid=0x00007f6b24bdd000 nid=0x4570 waiting on condition [0x00007f6afeac9000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
    at java.lang.Thread.sleep(Native Method)
    at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:104)

   Locked ownable synchronizers:
    - None

"Service Thread" #8 daemon prio=9 os_prio=0 tid=0x00007f6b241c0800 nid=0x456b runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"C1 CompilerThread1" #7 daemon prio=9 os_prio=0 tid=0x00007f6b241bb000 nid=0x456a waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 tid=0x00007f6b241b9000 nid=0x4569 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"Signal Dispatcher" #5 daemon prio=9 os_prio=0 tid=0x00007f6b24161000 nid=0x4568 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"Surrogate Locker Thread (Concurrent GC)" #4 daemon prio=9 os_prio=0 tid=0x00007f6b2415f000 nid=0x4567 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f6b24128000 nid=0x4566 in Object.wait() [0x00007f6affffe000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00000000ca6876e0> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
    - locked <0x00000000ca6876e0> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

   Locked ownable synchronizers:
    - None

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f6b24125800 nid=0x4565 in Object.wait() [0x00007f6b0c10a000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00000000ca687710> (a java.lang.ref.Reference$Lock)
    at java.lang.Object.wait(Object.java:502)
    at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
    - locked <0x00000000ca687710> (a java.lang.ref.Reference$Lock)
    at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

   Locked ownable synchronizers:
    - None

"main" #1 prio=5 os_prio=0 tid=0x00007f6b2400c800 nid=0x4560 runnable [0x00007f6b2a442000]
   java.lang.Thread.State: RUNNABLE
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1254)
    at java.net.SocketPermission.getIP(SocketPermission.java:789)
    at java.net.SocketPermission.impliesIgnoreMask(SocketPermission.java:912)
    at java.net.SocketPermission.implies(SocketPermission.java:846)
    at java.security.AccessControlContext.checkPermission2(AccessControlContext.java:515)
    at java.security.AccessControlContext.checkPermission(AccessControlContext.java:481)
    at java.security.AccessController.checkPermission(AccessController.java:884)
    at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
    at java.lang.SecurityManager.checkConnect(SecurityManager.java:1048)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1269)
    at java.net.InetAddress.getAllByName(InetAddress.java:1193)
    at java.net.InetAddress.getAllByName(InetAddress.java:1127)
    at java.net.InetAddress.getByName(InetAddress.java:1077)
    at java.net.InetSocketAddress.<init>(InetSocketAddress.java:220)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
    - locked <0x00000000c38521d0> (a sun.net.www.http.HttpClient)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
    at sun.net.www.http.HttpClient.New(HttpClient.java:339)
    at sun.net.www.http.HttpClient.New(HttpClient.java:357)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
    at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1040)
    at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1038)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1037)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
    - locked <0x00000000c384fab0> (a sun.net.www.protocol.http.HttpURLConnection)
    at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91)
    at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1484)
    at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1482)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1481)
    - locked <0x00000000c384fab0> (a sun.net.www.protocol.http.HttpURLConnection)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
    at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:399)
    at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285)
    at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252)
    at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684)
    at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
    at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681)
    at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:411)
    at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:311)
    at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.getStageLibraryManifestJson(ClassLoaderStageLibraryTask.java:1089)
    at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.getRepositoryManifestList(ClassLoaderStageLibraryTask.java:986)
    at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.initTask(ClassLoaderStageLibraryTask.java:346)
    at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:62)
    - locked <0x00000000ca696e60> (a com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask)
    at com.streamsets.datacollector.task.CompositeTask.initTask(CompositeTask.java:44)
    at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:62)
    - locked <0x00000000ca72fe50> (a com.streamsets.datacollector.main.PipelineTask)
    at com.streamsets.datacollector.task.TaskWrapper.init(TaskWrapper.java:40)
    at com.streamsets.datacollector.main.Main.lambda$doMain$1(Main.java:123)
    at com.streamsets.datacollector.main.Main$$Lambda$33/1103243338.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.streamsets.datacollector.security.SecurityUtil.doAs(SecurityUtil.java:92)
    at com.streamsets.datacollector.main.Main.doMain(Main.java:158)
    at com.streamsets.datacollector.main.DataCollectorMain.main(DataCollectorMain.java:53)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.streamsets.pipeline.BootstrapMain.bootstrap(BootstrapMain.java:351)
    at com.streamsets.pipeline.BootstrapMain.main(BootstrapMain.java:110)

   Locked ownable synchronizers:
    - None

"VM Thread" os_prio=0 tid=0x00007f6b2411b800 nid=0x4564 runnable 

"Gang worker#0 (Parallel GC Threads)" os_prio=0 tid=0x00007f6b2401e000 nid=0x4561 runnable 

"Gang worker#1 (Parallel GC Threads)" os_prio=0 tid=0x00007f6b2401f800 nid=0x4562 runnable 

"Concurrent Mark-Sweep GC Thread" os_prio=0 tid=0x00007f6b2404d000 nid=0x4563 runnable 

"VM Periodic Task Thread" os_prio=0 tid=0x00007f6b241c3800 nid=0x456c waiting on condition 

JNI global references: 369

Cannot start SDC 3.9.0

Hi,

I've upgraded SDC from 3.8.2 to 3.9.0 but after that the service doesn't start. I can see on gc.log that maybe something related to java memory issues.

Java HotSpot(TM) 64-Bit Server VM (25.201-b09) for linux-amd64 JRE (1.8.0_201-b09), built on Dec 15 2018 11:35:59 by "java_re" with gcc 7.3.0
Memory: 4k page, physical 8009556k(7749060k free), swap 4194300k(4194300k free)
CommandLine flags: -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/sdc/sdc_heapdump_1561534689.hprof -XX:InitialHeapSize=1073741824 -XX:MaxHeapSize=1073741824 -XX:MaxNewSize=174485504 -XX:MaxTenuringThreshold=6 -XX:NewSize=174485504 -XX:OldPLABSize=16 -XX:OldSize=348971008 -XX:-OmitStackTraceInFastThrow -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC 
2019-06-26T09:38:50.464+0200: 37.826: [GC (Allocation Failure) 2019-06-26T09:38:50.465+0200: 37.826: [ParNew: 136320K->12245K(153344K), 0.0863098 secs] 136320K->12245K(1031552K), 0.0864338 secs] [Times: user=0.03 sys=0.00, real=0.09 secs] 
2019-06-26T09:39:23.982+0200: 71.344: [GC (Allocation Failure) 2019-06-26T09:39:23.982+0200: 71.344: [ParNew: 148565K->7820K(153344K), 0.1149114 secs] 148565K->14721K(1031552K), 0.1150215 secs] [Times: user=0.07 sys=0.00, real=0.12 secs] 
2019-06-26T09:39:50.059+0200: 97.420: [GC (Allocation Failure) 2019-06-26T09:39:50.059+0200: 97.420: [ParNew: 144140K->4291K(153344K), 0.0230283 secs] 151041K->11192K(1031552K), 0.0231339 secs] [Times: user=0.01 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:01.450+0200: 108.811: [GC (Allocation Failure) 2019-06-26T09:40:01.450+0200: 108.811: [ParNew: 140611K->13047K(153344K), 0.0280423 secs] 147512K->19948K(1031552K), 0.0281406 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:02.463+0200: 109.825: [GC (Allocation Failure) 2019-06-26T09:40:02.463+0200: 109.825: [ParNew: 149367K->7162K(153344K), 0.0591644 secs] 156268K->22446K(1031552K), 0.0592586 secs] [Times: user=0.07 sys=0.00, real=0.06 secs] 
2019-06-26T09:40:03.035+0200: 110.396: [GC (Allocation Failure) 2019-06-26T09:40:03.035+0200: 110.396: [ParNew: 143482K->3056K(153344K), 0.0211872 secs] 158766K->18340K(1031552K), 0.0212620 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:05.485+0200: 112.846: [GC (Allocation Failure) 2019-06-26T09:40:05.485+0200: 112.846: [ParNew: 139376K->3838K(153344K), 0.0229737 secs] 154660K->19122K(1031552K), 0.0230716 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:06.486+0200: 113.848: [GC (Allocation Failure) 2019-06-26T09:40:06.486+0200: 113.848: [ParNew: 140158K->8095K(153344K), 0.0634246 secs] 155442K->23379K(1031552K), 0.0635210 secs] [Times: user=0.04 sys=0.00, real=0.06 secs] 
2019-06-26T09:40:06.561+0200: 113.923: [GC (CMS Initial Mark) [1 CMS-initial-mark: 15284K(878208K)] 31518K(1031552K), 0.0224020 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:06.584+0200: 113.945: [CMS-concurrent-mark-start]
2019-06-26T09:40:06.630+0200: 113.992: [CMS-concurrent-mark: 0.046/0.046 secs] [Times: user=0.08 sys=0.00, real=0.05 secs] 
2019-06-26T09:40:06.630+0200: 113.992: [CMS-concurrent-preclean-start]
2019-06-26T09:40:06.633+0200: 113.994: [CMS-concurrent-preclean: 0.003/0.003 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:06.633+0200: 113.994: [CMS-concurrent-abortable-preclean-start]
2019-06-26T09:40:07.173+0200: 114.535: [GC (Allocation Failure) 2019-06-26T09:40:07.173+0200: 114.535: [ParNew2019-06-26T09:40:07.203+0200: 114.565: [CMS-concurrent-abortable-preclean: 0.335/0.571 secs] [Times: user=0.63 sys=0.00, real=0.57 secs] 
: 144415K->8752K(153344K), 0.0378657 secs] 159699K->24036K(1031552K), 0.0379600 secs] [Times: user=0.03 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:07.243+0200: 114.605: [GC (CMS Final Remark) [YG occupancy: 16902 K (153344 K)]2019-06-26T09:40:07.244+0200: 114.605: [Rescan (parallel) , 0.0258777 secs]2019-06-26T09:40:07.269+0200: 114.631: [weak refs processing, 0.0000749 secs]2019-06-26T09:40:07.270+0200: 114.631: [class unloading, 0.0050736 secs]2019-06-26T09:40:07.275+0200: 114.636: [scrub symbol table, 0.0030306 secs]2019-06-26T09:40:07.278+0200: 114.639: [scrub string table, 0.0008257 secs][1 CMS-remark: 15284K(878208K)] 32186K(1031552K), 0.0355618 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:07.289+0200: 114.651: [CMS-concurrent-sweep-start]
2019-06-26T09:40:07.297+0200: 114.659: [CMS-concurrent-sweep: 0.008/0.008 secs] [Times: user=0.01 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:07.297+0200: 114.659: [CMS-concurrent-reset-start]
2019-06-26T09:40:07.326+0200: 114.687: [CMS-concurrent-reset: 0.018/0.029 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:07.725+0200: 115.087: [GC (Allocation Failure) 2019-06-26T09:40:07.725+0200: 115.087: [ParNew: 145072K->10610K(153344K), 0.0289038 secs] 160324K->25862K(1031552K), 0.0290027 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:08.212+0200: 115.574: [GC (Allocation Failure) 2019-06-26T09:40:08.212+0200: 115.574: [ParNew: 146930K->11709K(153344K), 0.0390318 secs] 162182K->28090K(1031552K), 0.0391571 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:08.673+0200: 116.034: [GC (Allocation Failure) 2019-06-26T09:40:08.673+0200: 116.034: [ParNew: 148029K->11129K(153344K), 0.0402671 secs] 164410K->28185K(1031552K), 0.0403611 secs] [Times: user=0.03 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:09.165+0200: 116.526: [GC (Allocation Failure) 2019-06-26T09:40:09.165+0200: 116.526: [ParNew: 147449K->10823K(153344K), 0.0295319 secs] 164505K->28409K(1031552K), 0.0296430 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:09.537+0200: 116.898: [GC (Allocation Failure) 2019-06-26T09:40:09.537+0200: 116.898: [ParNew: 147143K->7257K(153344K), 0.0409611 secs] 164729K->29869K(1031552K), 0.0410538 secs] [Times: user=0.05 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:10.252+0200: 117.613: [GC (Allocation Failure) 2019-06-26T09:40:10.252+0200: 117.613: [ParNew: 143577K->6902K(153344K), 0.0487168 secs] 166189K->30034K(1031552K), 0.0488106 secs] [Times: user=0.05 sys=0.00, real=0.05 secs] 
2019-06-26T09:40:19.949+0200: 127.311: [GC (Allocation Failure) 2019-06-26T09:40:19.950+0200: 127.311: [ParNew: 143222K->11748K(153344K), 0.0436810 secs] 166354K->35323K(1031552K), 0.0438189 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:20.595+0200: 127.956: [GC (Allocation Failure) 2019-06-26T09:40:20.595+0200: 127.956: [ParNew: 148068K->12469K(153344K), 0.0436302 secs] 171643K->37068K(1031552K), 0.0437625 secs] [Times: user=0.04 sys=0.00, real=0.05 secs] 
2019-06-26T09:40:20.991+0200: 128.353: [GC (Allocation Failure) 2019-06-26T09:40:20.991+0200: 128.353: [ParNew: 148789K->9785K(153344K), 0.0323473 secs] 173388K->34384K(1031552K), 0.0324717 secs] [Times: user=0.02 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:21.272+0200: 128.633: [GC (Allocation Failure) 2019-06-26T09:40:21.272+0200: 128.633: [ParNew: 146105K->9338K(153344K), 0.0200447 secs] 170704K->34602K(1031552K), 0.0201699 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:21.707+0200: 129.068: [GC (Allocation Failure) 2019-06-26T09:40:21.707+0200: 129.068: [ParNew: 145658K->10012K(153344K), 0.0286655 secs] 170922K->35720K(1031552K), 0.0287987 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:23.251+0200: 130.613: [GC (Allocation Failure) 2019-06-26T09:40:23.251+0200: 130.613: [ParNew: 146332K->12312K(153344K), 0.0270685 secs] 172040K->39280K(1031552K), 0.0272055 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:25.320+0200: 132.682: [GC (Allocation Failure) 2019-06-26T09:40:25.320+0200: 132.682: [ParNew: 148632K->12318K(153344K), 0.0357669 secs] 175600K->42343K(1031552K), 0.0358996 secs] [Times: user=0.07 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:25.568+0200: 132.930: [GC (Allocation Failure) 2019-06-26T09:40:25.568+0200: 132.930: [ParNew: 148638K->7849K(153344K), 0.0143115 secs] 178663K->38629K(1031552K), 0.0144382 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:31.386+0200: 138.748: [GC (Allocation Failure) 2019-06-26T09:40:31.386+0200: 138.748: [ParNew: 144163K->12286K(153344K), 0.0169158 secs] 174943K->43767K(1031552K), 0.0170368 secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:31.685+0200: 139.047: [GC (Allocation Failure) 2019-06-26T09:40:31.686+0200: 139.047: [ParNew: 148606K->9860K(153344K), 0.0301863 secs] 180087K->41830K(1031552K), 0.0303219 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:31.923+0200: 139.284: [GC (Allocation Failure) 2019-06-26T09:40:31.923+0200: 139.284: [ParNew: 146180K->10177K(153344K), 0.0300010 secs] 178150K->42770K(1031552K), 0.0301394 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:32.166+0200: 139.528: [GC (Allocation Failure) 2019-06-26T09:40:32.167+0200: 139.528: [ParNew: 146497K->10337K(153344K), 0.0187723 secs] 179090K->44185K(1031552K), 0.0188992 secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:32.480+0200: 139.842: [GC (Allocation Failure) 2019-06-26T09:40:32.480+0200: 139.842: [ParNew: 146657K->10576K(153344K), 0.0175061 secs] 180505K->46028K(1031552K), 0.0176373 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:35.179+0200: 142.540: [GC (Allocation Failure) 2019-06-26T09:40:35.179+0200: 142.540: [ParNew: 146896K->13689K(153344K), 0.0409025 secs] 182348K->49981K(1031552K), 0.0410205 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:35.459+0200: 142.820: [GC (Allocation Failure) 2019-06-26T09:40:35.459+0200: 142.820: [ParNew: 150009K->12614K(153344K), 0.0199374 secs] 186301K->51418K(1031552K), 0.0200658 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:38.059+0200: 145.421: [GC (Allocation Failure) 2019-06-26T09:40:38.059+0200: 145.421: [ParNew: 148934K->13399K(153344K), 0.0210353 secs] 187738K->52203K(1031552K), 0.0211810 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:43.064+0200: 150.426: [GC (Allocation Failure) 2019-06-26T09:40:43.064+0200: 150.426: [ParNew: 149719K->13295K(153344K), 0.0249492 secs] 188523K->54550K(1031552K), 0.0250791 secs] [Times: user=0.05 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:43.340+0200: 150.702: [GC (Allocation Failure) 2019-06-26T09:40:43.341+0200: 150.702: [ParNew: 149615K->8115K(153344K), 0.0202175 secs] 190870K->53376K(1031552K), 0.0203387 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:43.366+0200: 150.727: [GC (CMS Initial Mark) [1 CMS-initial-mark: 45261K(878208K)] 58813K(1031552K), 0.0068261 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:43.373+0200: 150.734: [CMS-concurrent-mark-start]
2019-06-26T09:40:43.450+0200: 150.812: [CMS-concurrent-mark: 0.078/0.078 secs] [Times: user=0.16 sys=0.00, real=0.08 secs] 
2019-06-26T09:40:43.450+0200: 150.812: [CMS-concurrent-preclean-start]
2019-06-26T09:40:43.453+0200: 150.815: [CMS-concurrent-preclean: 0.003/0.003 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:43.453+0200: 150.815: [CMS-concurrent-abortable-preclean-start]
2019-06-26T09:40:43.585+0200: 150.946: [GC (Allocation Failure) 2019-06-26T09:40:43.585+0200: 150.946: [ParNew: 144435K->7439K(153344K), 0.0119117 secs] 189696K->52701K(1031552K), 0.0120142 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:44.402+0200: 151.764: [GC (Allocation Failure) 2019-06-26T09:40:44.402+0200: 151.764: [ParNew: 143759K->10470K(153344K), 0.0138868 secs] 189021K->55731K(1031552K), 0.0140334 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:44.417+0200: 151.778: [CMS-concurrent-abortable-preclean: 0.396/0.963 secs] [Times: user=1.22 sys=0.00, real=0.97 secs] 
2019-06-26T09:40:44.418+0200: 151.779: [GC (CMS Final Remark) [YG occupancy: 13192 K (153344 K)]2019-06-26T09:40:44.418+0200: 151.779: [Rescan (parallel) , 0.0062470 secs]2019-06-26T09:40:44.424+0200: 151.785: [weak refs processing, 0.0000659 secs]2019-06-26T09:40:44.424+0200: 151.785: [class unloading, 0.0078585 secs]2019-06-26T09:40:44.432+0200: 151.793: [scrub symbol table, 0.0040422 secs]2019-06-26T09:40:44.436+0200: 151.797: [scrub string table, 0.0010877 secs][1 CMS-remark: 45261K(878208K)] 58454K(1031552K), 0.0197323 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:44.438+0200: 151.799: [CMS-concurrent-sweep-start]
2019-06-26T09:40:44.469+0200: 151.831: [CMS-concurrent-sweep: 0.032/0.032 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:44.469+0200: 151.831: [CMS-concurrent-reset-start]
2019-06-26T09:40:44.473+0200: 151.834: [CMS-concurrent-reset: 0.003/0.003 secs] [Times: user=0.01 sys=0.00, real=0.00 secs] 
2019-06-26T09:40:47.980+0200: 155.341: [GC (Allocation Failure) 2019-06-26T09:40:47.980+0200: 155.341: [ParNew: 146790K->10524K(153344K), 0.0185544 secs] 191767K->56403K(1031552K), 0.0187028 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:48.232+0200: 155.594: [GC (Allocation Failure) 2019-06-26T09:40:48.232+0200: 155.594: [ParNew: 146844K->7002K(153344K), 0.0159609 secs] 192723K->54194K(1031552K), 0.0160909 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:48.515+0200: 155.876: [GC (Allocation Failure) 2019-06-26T09:40:48.515+0200: 155.876: [ParNew: 143322K->10230K(153344K), 0.0166830 secs] 190514K->59060K(1031552K), 0.0168019 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:48.812+0200: 156.173: [GC (Allocation Failure) 2019-06-26T09:40:48.812+0200: 156.173: [ParNew: 146550K->8420K(153344K), 0.0203064 secs] 195380K->57741K(1031552K), 0.0204353 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:49.074+0200: 156.436: [GC (Allocation Failure) 2019-06-26T09:40:49.074+0200: 156.436: [ParNew: 144740K->8350K(153344K), 0.0188808 secs] 194061K->58300K(1031552K), 0.0190149 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:53.186+0200: 160.547: [GC (Allocation Failure) 2019-06-26T09:40:53.186+0200: 160.547: [ParNew: 144670K->9929K(153344K), 0.0178093 secs] 194620K->61153K(1031552K), 0.0179394 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:40:55.021+0200: 162.382: [GC (Allocation Failure) 2019-06-26T09:40:55.021+0200: 162.382: [ParNew: 146249K->10023K(153344K), 0.0342168 secs] 197473K->62884K(1031552K), 0.0343433 secs] [Times: user=0.07 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:58.877+0200: 166.239: [GC (Allocation Failure) 2019-06-26T09:40:58.877+0200: 166.239: [ParNew: 146343K->10757K(153344K), 0.0266899 secs] 199204K->64189K(1031552K), 0.0268154 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:40:59.216+0200: 166.577: [GC (Allocation Failure) 2019-06-26T09:40:59.216+0200: 166.577: [ParNew: 147077K->10578K(153344K), 0.0337889 secs] 200509K->64983K(1031552K), 0.0339147 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 
2019-06-26T09:40:59.475+0200: 166.836: [GC (Allocation Failure) 2019-06-26T09:40:59.475+0200: 166.836: [ParNew: 146898K->10449K(153344K), 0.0308185 secs] 201303K->65591K(1031552K), 0.0309496 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:40:59.855+0200: 167.216: [GC (Allocation Failure) 2019-06-26T09:40:59.855+0200: 167.216: [ParNew: 146769K->11019K(153344K), 0.0342162 secs] 201911K->67110K(1031552K), 0.0343465 secs] [Times: user=0.04 sys=0.00, real=0.04 secs] 
2019-06-26T09:41:00.115+0200: 167.477: [GC (Allocation Failure) 2019-06-26T09:41:00.115+0200: 167.477: [ParNew: 147336K->10448K(153344K), 0.0172150 secs] 203426K->67660K(1031552K), 0.0173502 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:01.984+0200: 169.346: [GC (Allocation Failure) 2019-06-26T09:41:01.985+0200: 169.346: [ParNew: 146768K->11447K(153344K), 0.0222017 secs] 203980K->70213K(1031552K), 0.0223227 secs] [Times: user=0.05 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:02.214+0200: 169.575: [GC (Allocation Failure) 2019-06-26T09:41:02.214+0200: 169.575: [ParNew: 147767K->6925K(153344K), 0.0172119 secs] 206533K->67457K(1031552K), 0.0173520 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:02.427+0200: 169.788: [GC (Allocation Failure) 2019-06-26T09:41:02.427+0200: 169.788: [ParNew: 143245K->5845K(153344K), 0.0141443 secs] 203777K->66988K(1031552K), 0.0142804 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:03.446+0200: 170.807: [GC (Allocation Failure) 2019-06-26T09:41:03.446+0200: 170.807: [ParNew: 142165K->11712K(153344K), 0.0150314 secs] 203308K->73305K(1031552K), 0.0151429 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:08.352+0200: 175.714: [GC (Allocation Failure) 2019-06-26T09:41:08.352+0200: 175.714: [ParNew: 148032K->10805K(153344K), 0.0191556 secs] 209625K->73712K(1031552K), 0.0193803 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:08.627+0200: 175.989: [GC (Allocation Failure) 2019-06-26T09:41:08.627+0200: 175.989: [ParNew: 147125K->11424K(153344K), 0.0155996 secs] 210032K->75312K(1031552K), 0.0157533 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:08.851+0200: 176.213: [GC (Allocation Failure) 2019-06-26T09:41:08.851+0200: 176.213: [ParNew: 147744K->7348K(153344K), 0.0159758 secs] 211632K->72884K(1031552K), 0.0161171 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:09.083+0200: 176.445: [GC (Allocation Failure) 2019-06-26T09:41:09.083+0200: 176.445: [ParNew: 143668K->6797K(153344K), 0.0125130 secs] 209204K->72782K(1031552K), 0.0126500 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:09.471+0200: 176.833: [GC (Allocation Failure) 2019-06-26T09:41:09.471+0200: 176.833: [ParNew: 143117K->7801K(153344K), 0.0134573 secs] 209102K->74208K(1031552K), 0.0136030 secs] [Times: user=0.02 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:09.685+0200: 177.046: [GC (Allocation Failure) 2019-06-26T09:41:09.685+0200: 177.046: [ParNew: 144121K->7923K(153344K), 0.0140758 secs] 210528K->75147K(1031552K), 0.0142106 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:09.905+0200: 177.266: [GC (Allocation Failure) 2019-06-26T09:41:09.905+0200: 177.266: [ParNew: 144243K->7461K(153344K), 0.0185685 secs] 211467K->76528K(1031552K), 0.0187047 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:11.066+0200: 178.427: [GC (Allocation Failure) 2019-06-26T09:41:11.066+0200: 178.427: [ParNew: 143781K->5783K(153344K), 0.0156430 secs] 212848K->75663K(1031552K), 0.0157626 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
2019-06-26T09:41:17.091+0200: 184.452: [GC (Allocation Failure) 2019-06-26T09:41:17.091+0200: 184.452: [ParNew: 142103K->8812K(153344K), 0.0213777 secs] 211983K->79053K(1031552K), 0.0215127 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:17.402+0200: 184.764: [GC (Allocation Failure) 2019-06-26T09:41:17.402+0200: 184.764: [ParNew: 145132K->10819K(153344K), 0.0148629 secs] 215373K->81788K(1031552K), 0.0149911 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:17.715+0200: 185.077: [GC (Allocation Failure) 2019-06-26T09:41:17.715+0200: 185.077: [ParNew: 147139K->8484K(153344K), 0.0174850 secs] 218108K->80236K(1031552K), 0.0176207 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:18.035+0200: 185.396: [GC (Allocation Failure) 2019-06-26T09:41:18.035+0200: 185.396: [ParNew: 144804K->12105K(153344K), 0.0186439 secs] 216556K->84164K(1031552K), 0.0187788 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:18.301+0200: 185.662: [GC (Allocation Failure) 2019-06-26T09:41:18.301+0200: 185.662: [ParNew: 148425K->12995K(153344K), 0.0218518 secs] 220484K->86356K(1031552K), 0.0219875 secs] [Times: user=0.05 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:18.634+0200: 185.996: [GC (Allocation Failure) 2019-06-26T09:41:18.634+0200: 185.996: [ParNew: 149315K->14491K(153344K), 0.0283266 secs] 222676K->90233K(1031552K), 0.0284645 secs] [Times: user=0.05 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:18.933+0200: 186.295: [GC (Allocation Failure) 2019-06-26T09:41:18.933+0200: 186.295: [ParNew: 150811K->15713K(153344K), 0.0303841 secs] 226553K->94447K(1031552K), 0.0305207 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:19.244+0200: 186.605: [GC (Allocation Failure) 2019-06-26T09:41:19.244+0200: 186.605: [ParNew: 152033K->14779K(153344K), 0.0217431 secs] 230767K->96395K(1031552K), 0.0218754 secs] [Times: user=0.04 sys=0.00, real=0.03 secs] 
2019-06-26T09:41:19.555+0200: 186.917: [GC (Allocation Failure) 2019-06-26T09:41:19.556+0200: 186.917: [ParNew: 151099K->13846K(153344K), 0.0209353 secs] 232715K->97223K(1031552K), 0.0210718 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] 
2019-06-26T09:41:19.848+0200: 187.210: [GC (Allocation Failure) 2019-06-26T09:41:19.848+0200: 187.210: [ParNew: 150166K->15004K(153344K), 0.0223914 secs] 233543K->101502K(1031552K), 0.0225120 secs] [Times: user=0.05 sys=0.00, real=0.02 secs]

Here the server conf: OS: Centos 7 64bit RAM: 8GB limits: ulimit -n 33000

Here the thread dump, captured with command sudo -u sdc jstack -l <pid> > /tmp/threadDump.txt

2019-06-26 18:01:49

Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.201-b09 mixed mode):

"Attach Listener" #107 daemon prio=9 os_prio=0 tid=0x00007f6ae8001000 nid=0x55db waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"FileWatchdog" #10 daemon prio=5 os_prio=0 tid=0x00007f6b24bdd000 nid=0x4570 waiting on condition [0x00007f6afeac9000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
    at java.lang.Thread.sleep(Native Method)
    at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:104)

   Locked ownable synchronizers:
    - None

"Service Thread" #8 daemon prio=9 os_prio=0 tid=0x00007f6b241c0800 nid=0x456b runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"C1 CompilerThread1" #7 daemon prio=9 os_prio=0 tid=0x00007f6b241bb000 nid=0x456a waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 tid=0x00007f6b241b9000 nid=0x4569 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"Signal Dispatcher" #5 daemon prio=9 os_prio=0 tid=0x00007f6b24161000 nid=0x4568 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"Surrogate Locker Thread (Concurrent GC)" #4 daemon prio=9 os_prio=0 tid=0x00007f6b2415f000 nid=0x4567 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
    - None

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f6b24128000 nid=0x4566 in Object.wait() [0x00007f6affffe000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00000000ca6876e0> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
    - locked <0x00000000ca6876e0> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

   Locked ownable synchronizers:
    - None

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f6b24125800 nid=0x4565 in Object.wait() [0x00007f6b0c10a000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00000000ca687710> (a java.lang.ref.Reference$Lock)
    at java.lang.Object.wait(Object.java:502)
    at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
    - locked <0x00000000ca687710> (a java.lang.ref.Reference$Lock)
    at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

   Locked ownable synchronizers:
    - None

"main" #1 prio=5 os_prio=0 tid=0x00007f6b2400c800 nid=0x4560 runnable [0x00007f6b2a442000]
   java.lang.Thread.State: RUNNABLE
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1254)
    at java.net.SocketPermission.getIP(SocketPermission.java:789)
    at java.net.SocketPermission.impliesIgnoreMask(SocketPermission.java:912)
    at java.net.SocketPermission.implies(SocketPermission.java:846)
    at java.security.AccessControlContext.checkPermission2(AccessControlContext.java:515)
    at java.security.AccessControlContext.checkPermission(AccessControlContext.java:481)
    at java.security.AccessController.checkPermission(AccessController.java:884)
    at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
    at java.lang.SecurityManager.checkConnect(SecurityManager.java:1048)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1269)
    at java.net.InetAddress.getAllByName(InetAddress.java:1193)
    at java.net.InetAddress.getAllByName(InetAddress.java:1127)
    at java.net.InetAddress.getByName(InetAddress.java:1077)
    at java.net.InetSocketAddress.<init>(InetSocketAddress.java:220)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
    - locked <0x00000000c38521d0> (a sun.net.www.http.HttpClient)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
    at sun.net.www.http.HttpClient.New(HttpClient.java:339)
    at sun.net.www.http.HttpClient.New(HttpClient.java:357)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
    at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1040)
    at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1038)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1037)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
    - locked <0x00000000c384fab0> (a sun.net.www.protocol.http.HttpURLConnection)
    at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91)
    at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1484)
    at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1482)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1481)
    - locked <0x00000000c384fab0> (a sun.net.www.protocol.http.HttpURLConnection)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
    at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:399)
    at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285)
    at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252)
    at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684)
    at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
    at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681)
    at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:411)
    at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:311)
    at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.getStageLibraryManifestJson(ClassLoaderStageLibraryTask.java:1089)
    at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.getRepositoryManifestList(ClassLoaderStageLibraryTask.java:986)
    at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.initTask(ClassLoaderStageLibraryTask.java:346)
    at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:62)
    - locked <0x00000000ca696e60> (a com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask)
    at com.streamsets.datacollector.task.CompositeTask.initTask(CompositeTask.java:44)
    at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:62)
    - locked <0x00000000ca72fe50> (a com.streamsets.datacollector.main.PipelineTask)
    at com.streamsets.datacollector.task.TaskWrapper.init(TaskWrapper.java:40)
    at com.streamsets.datacollector.main.Main.lambda$doMain$1(Main.java:123)
    at com.streamsets.datacollector.main.Main$$Lambda$33/1103243338.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.streamsets.datacollector.security.SecurityUtil.doAs(SecurityUtil.java:92)
    at com.streamsets.datacollector.main.Main.doMain(Main.java:158)
    at com.streamsets.datacollector.main.DataCollectorMain.main(DataCollectorMain.java:53)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.streamsets.pipeline.BootstrapMain.bootstrap(BootstrapMain.java:351)
    at com.streamsets.pipeline.BootstrapMain.main(BootstrapMain.java:110)

   Locked ownable synchronizers:
    - None

"VM Thread" os_prio=0 tid=0x00007f6b2411b800 nid=0x4564 runnable 

"Gang worker#0 (Parallel GC Threads)" os_prio=0 tid=0x00007f6b2401e000 nid=0x4561 runnable 

"Gang worker#1 (Parallel GC Threads)" os_prio=0 tid=0x00007f6b2401f800 nid=0x4562 runnable 

"Concurrent Mark-Sweep GC Thread" os_prio=0 tid=0x00007f6b2404d000 nid=0x4563 runnable 

"VM Periodic Task Thread" os_prio=0 tid=0x00007f6b241c3800 nid=0x456c waiting on condition 

JNI global references: 369

UPDATE: The service has finally started, after 45 minutes! checking sdc.log I noticed a strong delay in the section "Reading Repository Manifest"

2019-06-27 09:46:36,155 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - -----------------------------------------------------------------
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - Build info:
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Version        : 3.9.0
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Date           : 2019-06-05T21:35Z
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Built by       : ubuntu
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Repo SHA       : 92ea14942c92010739f0d93782cca1abb62a2e0c
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Source MD5     : 532070114c8481a4635f48732a5a1f39
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - -----------------------------------------------------------------
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - Runtime info:
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Java version  : 1.8.0_201-b09
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   SDC ID        : 8b866309-9826-11e9-9a41-9586a5fb1716
2019-06-27 09:46:36,157 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Runtime dir   : /opt/streamsets-datacollector
2019-06-27 09:46:36,158 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Config dir    : /etc/sdc
2019-06-27 09:46:36,158 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Data dir      : /var/lib/sdc
2019-06-27 09:46:36,158 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Log dir       : /var/log/sdc
2019-06-27 09:46:36,158 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Extra Libs dir: /opt/streamsets-datacollector/streamsets-libs-extras
2019-06-27 09:46:36,158 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - -----------------------------------------------------------------
2019-06-27 09:46:36,158 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Security Manager : ENABLED, policy file: file:///etc/sdc/sdc-security.policy, implementation: java.lang.SecurityManager
2019-06-27 09:46:36,158 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - -----------------------------------------------------------------
2019-06-27 09:46:36,158 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - Starting ...
2019-06-27 09:46:36,163 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - -----------------------------------------------------------------
2019-06-27 09:46:36,163 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Kerberos enabled: false
2019-06-27 09:46:36,247 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main -   Unlimited cryptography enabled: true
2019-06-27 09:46:36,247 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - -----------------------------------------------------------------
2019-06-27 09:46:36,247 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Main - Starting ...
2019-06-27 09:46:36,252 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  ClassLoaderStageLibraryTask - Validating classpath of all stages
2019-06-27 09:46:37,541 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  ClassLoaderStageLibraryTask - Finished classpath validation in 1289 ms
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult - Validation results for streamsets-datacollector-azure-lib
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult - Detected colliding dependency versions:
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -   Dependency jackson have versions: 2.8.9, 2.9.5, 2.8.6, 1.9.13, 2.8.0, 1.9.2
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -     Version: 2.8.9
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-jaxrs-base-2.8.9.jar
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-databind-2.8.9.jar
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-jaxrs-json-provider-2.8.9.jar
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -     Version: 2.9.5
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-module-jaxb-annotations-2.9.5.jar
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -     Version: 2.8.6
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-core-2.8.6.jar
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -     Version: 1.9.13
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-mapper-asl-1.9.13.jar
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-core-asl-1.9.13.jar
2019-06-27 09:46:37,542 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -     Version: 2.8.0
2019-06-27 09:46:37,543 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-annotations-2.8.0.jar
2019-06-27 09:46:37,543 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -     Version: 1.9.2
2019-06-27 09:46:37,543 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-xc-1.9.2.jar
2019-06-27 09:46:37,543 [user:] [pipeline:] [runner:] [thread:main] [stage:] WARN  ClasspathValidatorResult -       file:/opt/streamsets-datacollector/streamsets-libs/streamsets-datacollector-azure-lib/lib/jackson-jaxrs-1.9.2.jar
2019-06-27 09:46:37,545 [user:] [pipeline:] [runner:] [thread:main] [stage:] ERROR ClassLoaderStageLibraryTask - The following stages have invalid classpath: streamsets-datacollector-azure-lib
2019-06-27 09:47:00,233 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  ClassLoaderStageLibraryTask - Loaded '65' libraries with a total of '345' stages, '0' lineage publishers, '3' services, '0' interceptors, '1' delegates and '6' credentialStores in '22670ms'
2019-06-27 09:47:03,544 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  ClassLoaderStageLibraryTask - Reading from Repository Manifest URL: http://archives.streamsets.com/datacollector/3.9.0/tarball/repository.manifest.json
2019-06-27 10:12:21,779 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  ClassLoaderStageLibraryTask - Reading from Repository Manifest URL: http://archives.streamsets.com/datacollector/3.9.0/tarball/enterprise/repository.manifest.json
2019-06-27 10:15:29,100 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  ClassLoaderStageLibraryTask - Reading from Repository Manifest URL: http://archives.streamsets.com/datacollector/3.9.0/legacy/repository.manifest.json
2019-06-27 10:27:40,031 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  LineagePublisherTaskImpl - No publishers configured
2019-06-27 10:27:45,362 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  StatsCollectorTask - Stats Collection, opted 'true, active 'true'
2019-06-27 10:27:45,401 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  AntennaDoctor - Initializing Antenna Doctor
2019-06-27 10:27:45,401 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  AntennaDoctorStorage - Repository location: /var/lib/sdc/antennadoctor
2019-06-27 10:27:45,523 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  AntennaDoctorEngine - Loaded new Antenna Doctor engine with 4 rules
2019-06-27 10:27:46,242 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Reflections - Reflections took 484 ms to scan 2 urls, producing 1445 keys and 2360 values
2019-06-27 10:27:48,639 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  Reflections - Reflections took 127 ms to scan 2 urls, producing 1445 keys and 2360 values
2019-06-27 10:27:49,182 [user:] [pipeline:] [runner:] [thread:main] [stage:] INFO  WebServerTask - Running on URI : 'http://test_machine:18630'