SQLException: No suitable driver

edit

The log shows: SQLException: No suitable driver when running a preview of a simple pipeline with a MariaDB database.

My StreamSets instance is a Docker one and it should access a MariaDB instance which is also a Docker one.

I checked that the MariaDB instance can be connected to by a client in the host. This to make sure I can access it with the same url from the host.

I understand the StreamSets instance cannot find the JDBC driver, possibly because some missing configuration around the JDBC Producer stage lib ? Or could it be a wrong url parameter ?

I created the following Docker Compose file:

version: "3.7"
services:
  streamsets:
    image: localhost:5000/streamsets:latest
    ports:
      - "18630:18630"
    networks:
      streamsets:
      common:
    volumes:
      - "sdc-data:/data"
      - "sdc-stagelibs:/opt/streamsets-datacollector-3.18.1/streamsets-libs"
      - "sdc-stagelibs-extra:/opt/stage-libs-extra"
      - "~/dev/docker/projects/streamsets/volumes/logs:/logs"
    environment:
      SDC_CONF_STREAMSETS_LIBRARIES_EXTRA_DIR: "/opt/stage-libs-extra"
      HOST_USER_ID: ${CURRENT_UID}
      HOST_GROUP_ID: ${CURRENT_GID}
    deploy:
      replicas: 1
      restart_policy:
        condition: any
        delay: 5s
        max_attempts: 3
        window: 10s
    healthcheck:
      test: curl --fail http://localhost:18630 || exit 1
      interval: 1m
      timeout: 3s
      retries: 3
volumes:
  sdc-data:
  sdc-stagelibs:
  sdc-stagelibs-extra:
networks:
  streamsets:
    name: streamsets
  common:
    external: true
    name: common

along with the volumes:

docker volume create --name sdc-data;
docker volume create --name sdc-stagelibs;
docker volume create --name sdc-stagelibs-extra;

I have installed some stage libs from the admin package manager and they are seen in:

bash-4.4$ ls -l /opt/streamsets-datacollector-3.18.1/streamsets-libs
total 28
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-basic-lib
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-dataformats-lib
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-dev-lib
drwxr-xr-x    3 sdc      sdc           4096 Sep 10 22:19 streamsets-datacollector-jdbc-lib
drwxr-xr-x    3 sdc      sdc           4096 Sep 10 21:49 streamsets-datacollector-mysql-binlog-lib
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-stats-lib
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-windows-lib
bash-4.4$ ls -l /opt/stage-libs-extra
total 0

Should the streamsets-datacollector-jdbc-lib stage lib be installed in the stage-libs-extra directory ? If so, why is it installed in the main stage libs directory ?

I note that in the StreamSets container the STREAMSETS_LIBRARIES_EXTRA_DIR variable is not the one I specified in the Docker Compose file:

bash-4.4$ echo $STREAMSETS_LIBRARIES_EXTRA_DIR
/opt/streamsets-datacollector-3.18.1/streamsets-libs-extras

UPDATE: I changed the volume for the stage libs extra directory and now use a host directory:

- "~/dev/docker/projects/streamsets/volumes/stage-libs-extra:/opt/stage-libs-extra"

I has the following user and permissions:

stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra
total 4
drwxrwxr-x 3 stephane stephane 4096 sept. 12 09:30 streamsets-datacollector-jdbc-lib
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra/
total 4
drwxrwxr-x 3 stephane stephane 4096 sept. 12 09:30 streamsets-datacollector-jdbc-lib
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra/streamsets-datacollector-jdbc-lib/
total 4
drwxrwxr-x 2 stephane stephane 4096 sept. 12 09:32 lib
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra/streamsets-datacollector-jdbc-lib/lib/
total 3980
-rw-rw-r-- 1 stephane stephane 1715434 sept. 12 09:30 mariadb-java-client-2.6.2-javadoc.jar
-rw-rw-r-- 1 stephane stephane 2356711 sept. 12 09:32 mysql-connector-java-8.0.19 ...