Ask Your Question
1

SQLException: No suitable driver

asked 2020-09-11 09:06:56 -0500

Stephane gravatar image

updated 2020-09-12 02:49:13 -0500

The log shows: SQLException: No suitable driver when running a preview of a simple pipeline with a MariaDB database.

My StreamSets instance is a Docker one and it should access a MariaDB instance which is also a Docker one.

I checked that the MariaDB instance can be connected to by a client in the host. This to make sure I can access it with the same url from the host.

I understand the StreamSets instance cannot find the JDBC driver, possibly because some missing configuration around the JDBC Producer stage lib ? Or could it be a wrong url parameter ?

I created the following Docker Compose file:

version: "3.7"
services:
  streamsets:
    image: localhost:5000/streamsets:latest
    ports:
      - "18630:18630"
    networks:
      streamsets:
      common:
    volumes:
      - "sdc-data:/data"
      - "sdc-stagelibs:/opt/streamsets-datacollector-3.18.1/streamsets-libs"
      - "sdc-stagelibs-extra:/opt/stage-libs-extra"
      - "~/dev/docker/projects/streamsets/volumes/logs:/logs"
    environment:
      SDC_CONF_STREAMSETS_LIBRARIES_EXTRA_DIR: "/opt/stage-libs-extra"
      HOST_USER_ID: ${CURRENT_UID}
      HOST_GROUP_ID: ${CURRENT_GID}
    deploy:
      replicas: 1
      restart_policy:
        condition: any
        delay: 5s
        max_attempts: 3
        window: 10s
    healthcheck:
      test: curl --fail http://localhost:18630 || exit 1
      interval: 1m
      timeout: 3s
      retries: 3
volumes:
  sdc-data:
  sdc-stagelibs:
  sdc-stagelibs-extra:
networks:
  streamsets:
    name: streamsets
  common:
    external: true
    name: common

along with the volumes:

docker volume create --name sdc-data;
docker volume create --name sdc-stagelibs;
docker volume create --name sdc-stagelibs-extra;

I have installed some stage libs from the admin package manager and they are seen in:

bash-4.4$ ls -l /opt/streamsets-datacollector-3.18.1/streamsets-libs
total 28
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-basic-lib
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-dataformats-lib
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-dev-lib
drwxr-xr-x    3 sdc      sdc           4096 Sep 10 22:19 streamsets-datacollector-jdbc-lib
drwxr-xr-x    3 sdc      sdc           4096 Sep 10 21:49 streamsets-datacollector-mysql-binlog-lib
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-stats-lib
drwxrwxr-x    3 sdc      sdc           4096 Sep 10 21:45 streamsets-datacollector-windows-lib
bash-4.4$ ls -l /opt/stage-libs-extra
total 0

Should the streamsets-datacollector-jdbc-lib stage lib be installed in the stage-libs-extra directory ? If so, why is it installed in the main stage libs directory ?

I note that in the StreamSets container the STREAMSETS_LIBRARIES_EXTRA_DIR variable is not the one I specified in the Docker Compose file:

bash-4.4$ echo $STREAMSETS_LIBRARIES_EXTRA_DIR
/opt/streamsets-datacollector-3.18.1/streamsets-libs-extras

UPDATE: I changed the volume for the stage libs extra directory and now use a host directory:

- "~/dev/docker/projects/streamsets/volumes/stage-libs-extra:/opt/stage-libs-extra"

I has the following user and permissions:

stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra
total 4
drwxrwxr-x 3 stephane stephane 4096 sept. 12 09:30 streamsets-datacollector-jdbc-lib
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra/
total 4
drwxrwxr-x 3 stephane stephane 4096 sept. 12 09:30 streamsets-datacollector-jdbc-lib
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra/streamsets-datacollector-jdbc-lib/
total 4
drwxrwxr-x 2 stephane stephane 4096 sept. 12 09:32 lib
stephane@stephane-pc:~/dev/docker/projects/streamsets$ ll volumes/stage-libs-extra/streamsets-datacollector-jdbc-lib/lib/
total 3980
-rw-rw-r-- 1 stephane stephane 1715434 sept. 12 09:30 mariadb-java-client-2.6.2-javadoc.jar
-rw-rw-r-- 1 stephane stephane 2356711 sept. 12 09:32 mysql-connector-java-8.0.19 ...
(more)
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2020-09-12 03:58:47 -0500

Stephane gravatar image

I could solve my issue by uploading the connector jar file with the admin panel, instead of copying the files at the command line.

By reading some troubleshouting instructions I understood the connector jar file needed to be added to each stage lib using it.

Here is how I proceeded:

Install the MySQL connector into the MySQL BinLog stage lib extra directory Log in the admin panel Click on the Package Manager icon At the bottom of the left pane, click on the External Libraries button On the top right of the page, click on the Install External Libraries icon In the dialog box, first select the stage library for which to install the connector Select the MySQL BinLog entry Browse to the connector jar file and upload it

A same jar file must be installed in each stage library that needs to use it

Repeat the operation for the JDBC stage lib extra directory

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-09-11 09:03:15 -0500

Seen: 285 times

Last updated: Sep 12