Ask Your Question
1

What can cause SDC GUI to slow down as pipelines are added?

asked 2017-09-08 13:18:52 -0500

mstang gravatar image

I have SDC running alone on a fairly large centos server (48 cores, 256GB memory). I'm noticing a big slowdown in the GUI as I add pipelines but I'm not seeing any errors as I tail sdc.log. Symptoms are very slow refresh and saves to pipeline changes, slow starting of pipelines.

I'm currently running SDC 2.7.1 and I've got 126 defined pipelines, 18 running pipelines all pretty simple with a MapRDB-JSON Origin (5 or six fields) -> MapRFS destination->Shell Executor (moving the files after they close) not a huge volume of data moving through the pipelines.

Is there something I should be adjusting in config (I've already bumped threads up to 200) or would I want to run more than one sdc instance on my server?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2017-09-11 08:55:35 -0500

Mufy gravatar image

Hello Mstang. Some obvious queries -

=> During the time of this slowness, have you taken a general look at the resource utilization on the node (top would be an easy place to look)? Any contentions or abnormal utilization with respect to CPU, memory, etc.? What processes are seen to top the chart? => How about disk I/Os during the same degradation window? => Having MapR-DB on the same node I'd safely assume a good part of the node memory automatically earmarked for that, unless manually tweaked. Not saying that this is the problem area, but something definitely worth considering.

Basically, we may be mislead with the symptom (SDC slowness) brought about by some other underlying issue. That's my line of attack for now.

edit flag offensive delete link more

Comments

This is a good point that I'll need to investigate, since this is on a MapR cluster node it will be sucking up resources that I don't carve out for the OS. In the mean time I rebuilt my process to run the same data through 3 pipelines instead of 30-40 and I no longer have any issue with slow gui.

mstang gravatar imagemstang ( 2017-09-12 13:00:03 -0500 )edit

My basic questions still stand though on what my expectations should be on pipelines -> cores/memory (understanding that this could vary greatly depending on what the pipelines are doing) and what parameters need to be adjusted as the number of pipelines increase.

mstang gravatar imagemstang ( 2017-09-12 13:04:40 -0500 )edit

Hmm, I'm inclined to think that launching a bunch of pipelines would have resulted in individual JVMs requiring their own "cut" of the resources. Reducing which gave you them the needed room to breathe.

Mufy gravatar imageMufy ( 2017-09-12 22:18:26 -0500 )edit

About recommendations on configurations to tune performance, let me have a look and see if I can find something useful for you.

Mufy gravatar imageMufy ( 2017-09-12 22:18:44 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-09-08 13:18:52 -0500

Seen: 167 times

Last updated: Sep 11 '17