Ask Your Question
1

streamsets-dc docker container exits on start-up

asked 2020-04-21 03:02:43 -0500

Paul Tipper gravatar image

I'm trying to start the streamsets-dc docker container, but the container is exiting on start-up. Here's what I see when I run docker ps -a:

18:06 $ docker ps -a
CONTAINER ID  IMAGE                                           COMMAND   CREATED        STATUS                    PORTS                             NAMES
ec48521394f6  docker.io/streamsets/datacollector:latest       dc        8 seconds ago  Exited (1) 7 seconds ago  0.0.0.0:18630->18630/tcp          streamsets-dc
82bdce9df314  localhost/identityserverui:latest                         2 months ago   Created                   0.0.0.0:5000-5001->5000-5001/tcp  identity-server_ui_1
6bc11a3bd09e  identityserverregistry.azurecr.io/idxapi:2.3.4            2 months ago   Created                   0.0.0.0:5000-5001->5000-5001/tcp  identity-server_api_1
469d193f09c5  localhost/identityserver:latest                           2 months ago   Created                   0.0.0.0:5000-5001->5000-5001/tcp  identity-server_idp_1
f2be3820eed9  docker.io/library/postgres:latest               postgres  2 months ago   Created                   0.0.0.0:5000-5001->5000-5001/tcp  identity-server_db_1
da3006a3c4b3  k8s.gcr.io/pause:3.1                                      2 months ago   Created                   0.0.0.0:5000-5001->5000-5001/tcp  d5a2b56c021b-infra

You can see that the streamsets-dc container exits immediately on start-up. If I run the docker logs streamsets-dc command, I get the following

Java 1.8 detected; adding $SDC_JAVA8_OPTS of "-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Djdk.nio.maxCachedBufferSize=262144" to $SDC_JAVA_OPTS
Configuration of maximum open file limit is too low: 1024 (expected at least 32768). Please consult https://goo.gl/6dmjXd

I'm attempting to start the container within a Fedora 31 VirtualBox virtual machine.

After Googling a bit, I saw lots of advice about raising the open file limit using ulimit, but tying ulimit -n in the virtual machine in which I'm trying to start this container returns a value of 524288, which is far in excess of the 32678 limit required by streamsets-dc.

I have to admit I'm not all that familiar with docker, but I would have assumed that the streamsets-dc container would have everything configured to just run out of the box.

Can anyone suggest what I need to do to get the container to run?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
2

answered 2020-04-21 07:11:36 -0500

Paul Tipper gravatar image

updated 2020-04-21 07:12:24 -0500

Finally managed to get the docker contain to install and run by typing the following command:

sudo docker run --ulimit nofile=90000:90000 --restart on-failure -p 18630:18630 -d --name sdc streamsets/datacollector

Apparently docker containers run with a default ulimit of 1024, but you can override this default from the command line using the --ulimit flag.

Also, because I was attempting to run the container within a Linux VM, I needed to run docker as sudo to give the build process adequate permissions to write to certain system directories.

My data collector instance is now running on my VM on port 18630.

edit flag offensive delete link more

Comments

1

Thanks for coming back and sharing the solution, Paul!

metadaddy gravatar imagemetadaddy ( 2020-04-21 09:17:32 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-04-21 03:02:43 -0500

Seen: 27 times

Last updated: Apr 21