SQLException: No suitable driver
The log shows: SQLException: No suitable driver
when running a preview of a simple pipeline with a MariaDB database.
My StreamSets instance is a Docker one and it should access a MariaDB instance which is also a Docker one.
I checked that the MariaDB instance can be connected to by a client in the host. This to make sure I can access it with the same url from the host.
I understand the StreamSets instance cannot find the JDBC driver, possibly because some missing configuration around the JDBC Producer stage lib ? Or could it be a wrong url parameter ?
I created the following Docker Compose file:
version: "3.7"
services:
streamsets:
image: localhost:5000/streamsets:latest
ports:
- "18630:18630"
networks:
streamsets:
common:
volumes:
- "sdc-data:/data"
- "sdc-stagelibs:/opt/streamsets-datacollector-3.18.1/streamsets-libs"
- "sdc-stagelibs-extra:/opt/stage-libs-extra"
- "~/dev/docker/projects/streamsets/volumes/logs:/logs"
environment:
SDC_CONF_STREAMSETS_LIBRARIES_EXTRA_DIR: "/opt/stage-libs-extra"
HOST_USER_ID: ${CURRENT_UID}
HOST_GROUP_ID: ${CURRENT_GID}
deploy:
replicas: 1
restart_policy:
condition: any
delay: 5s
max_attempts: 3
window: 10s
healthcheck:
test: curl --fail http://localhost:18630 || exit 1
interval: 1m
timeout: 3s
retries: 3
volumes:
sdc-data:
sdc-stagelibs:
sdc-stagelibs-extra:
networks:
streamsets:
name: streamsets
common:
external: true
name: common
along with the volumes:
docker volume create --name sdc-data;
docker volume create --name sdc-stagelibs;
docker volume create --name sdc-stagelibs-extra;
I have installed some stage libs from the admin package manager and they are seen in:
bash-4.4$ ls -l /opt/streamsets-datacollector-3.18.1/streamsets-libs
total 28
drwxrwxr-x 3 sdc sdc 4096 Sep 10 21:45 streamsets-datacollector-basic-lib
drwxrwxr-x 3 sdc sdc 4096 Sep 10 21:45 streamsets-datacollector-dataformats-lib
drwxrwxr-x 3 sdc sdc 4096 Sep 10 21:45 streamsets-datacollector-dev-lib
drwxr-xr-x 3 sdc sdc 4096 Sep 10 22:19 streamsets-datacollector-jdbc-lib
drwxr-xr-x 3 sdc sdc 4096 Sep 10 21:49 streamsets-datacollector-mysql-binlog-lib
drwxrwxr-x 3 sdc sdc 4096 Sep 10 21:45 streamsets-datacollector-stats-lib
drwxrwxr-x 3 sdc sdc 4096 Sep 10 21:45 streamsets-datacollector-windows-lib
bash-4.4$ ls -l /opt/stage-libs-extra
total 0
Should the streamsets-datacollector-jdbc-lib
stage lib be installed in the stage-libs-extra
directory ? If so, why is it installed in the main stage libs directory ?
I note that in the StreamSets container the STREAMSETS_LIBRARIES_EXTRA_DIR
variable is not the one I specified in the Docker Compose file:
bash-4.4$ echo $STREAMSETS_LIBRARIES_EXTRA_DIR
/opt/streamsets-datacollector-3.18.1/streamsets-libs-extras
UPDATE: I changed the volume for the stage libs extra directory and now use a host directory:
- "~/dev/docker/projects/streamsets/volumes/stage-libs-extra:/opt/stage-libs-extra"
I has the following user and permissions:
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra
total 4
drwxrwxr-x 3 stephane stephane 4096 sept. 12 09:30 streamsets-datacollector-jdbc-lib
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra/
total 4
drwxrwxr-x 3 stephane stephane 4096 sept. 12 09:30 streamsets-datacollector-jdbc-lib
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra/streamsets-datacollector-jdbc-lib/
total 4
drwxrwxr-x 2 stephane stephane 4096 sept. 12 09:32 lib
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra/streamsets-datacollector-jdbc-lib/lib/
total 3980
-rw-rw-r-- 1 stephane stephane 1715434 sept. 12 09:30 mariadb-java-client-2.6.2-javadoc.jar
-rw-rw-r-- 1 stephane stephane 2356711 sept. 12 09:32 mysql-connector-java-8.0.19 ...