Ask Your Question
1

I cannot connect to a Hive table using JDBC Query origin

asked 2017-09-21 21:45:05 -0500

hdelgad gravatar image

updated 2017-10-03 10:10:16 -0500

metadaddy gravatar image

I am trying to read a table so I can merge it with another one. in the JDBC connection string I am getting the following error:

JDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: com.zaxxer.hikari.pool.PoolInitializationException: Exception during pool initialization: org/apache/thrift/TException

This is the trace form sdc.log:

2017-10-02 11:00:48,368 [user:myuserid] [pipeline:HDT_TEST_UPDATE2/HDTTESTUPDATE2pipelineid] [runner:] [thread:webserver-798] INFO  StandaloneAndClusterRunnerProviderImpl - Pipeline execution mode is: STANDALONE 
2017-10-02 11:01:25,090 [user:myuserid] [pipeline:HDT_TEST_UPDATE2/HDTTESTUPDATE2pipelineid] [runner:] [thread:webserver-783] INFO  StandaloneRunner - Preparing to start pipeline 'HDTTESTUPDATE2pipelineid::0'
2017-10-02 11:01:25,112 [user:*myuserid] [pipeline:HDT_TEST_UPDATE2/HDTTESTUPDATE2pipelineid] [runner:] [thread:runner-pool-2-thread-19] INFO  StandaloneRunner - Starting pipeline HDTTESTUPDATE2pipelineid 0
2017-10-02 11:01:25,144 [user:*myuserid] [pipeline:HDT_TEST_UPDATE2/HDTTESTUPDATE2pipelineid] [runner:] [thread:runner-pool-2-thread-19] INFO  ProductionPipelineRunner - Adding error listeners0
2017-10-02 11:01:25,214 [user:*myuserid] [pipeline:HDT_TEST_UPDATE2/HDTTESTUPDATE2pipelineid] [runner:] [thread:ProductionPipelineRunnable-HDTTESTUPDATE2pipelineid-HDT_TEST_UPDATE2] INFO  JdbcUtil - Registered JDBC drivers:
2017-10-02 11:01:25,220 [user:*myuserid] [pipeline:HDT_TEST_UPDATE2/HDTTESTUPDATE2pipelineid] [runner:] [thread:ProductionPipelineRunnable-HDTTESTUPDATE2pipelineid-HDT_TEST_UPDATE2] INFO  JdbcUtil - Driver class org.apache.hive.jdbc.HiveDriver (version 2.1)
2017-10-02 11:01:25,229 [user:*myuserid] [pipeline:HDT_TEST_UPDATE2/HDTTESTUPDATE2pipelineid] [runner:] [thread:ProductionPipelineRunnable-HDTTESTUPDATE2pipelineid-HDT_TEST_UPDATE2] INFO  HikariDataSource - HikariPool-6 - is starting.
2017-10-02 11:01:25,237 [user:*myuserid] [pipeline:HDT_TEST_UPDATE2/HDTTESTUPDATE2pipelineid] [runner:] [thread:ProductionPipelineRunnable-HDTTESTUPDATE2pipelineid-HDT_TEST_UPDATE2] INFO  HikariPool - HikariPool-6 - is closing down.
2017-10-02 11:01:25,239 [user:*myuserid] [pipeline:HDT_TEST_UPDATE2/HDTTESTUPDATE2pipelineid] [runner:] [thread:ProductionPipelineRunnable-HDTTESTUPDATE2pipelineid-HDT_TEST_UPDATE2] ERROR JdbcUtil - Failed to initialize connection pool: {}
com.zaxxer.hikari.pool.PoolInitializationException: Exception during pool initialization: org/apache/thrift/TException
    at com.zaxxer.hikari.pool.HikariPool.initializeConnections(HikariPool.java:581)
    at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:152)
    at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:73)
    at com.streamsets.pipeline.lib.jdbc.JdbcUtil.createDataSourceForRead(JdbcUtil.java:630)
    at com.streamsets.pipeline.stage.origin.jdbc.JdbcSource.init(JdbcSource.java:211)
    at com.streamsets.pipeline.api.base.BaseStage.init(BaseStage.java:52)
    at com.streamsets.pipeline.configurablestage.DStage.init(DStage.java:40)
    at com.streamsets.datacollector.runner.StageRuntime.init(StageRuntime.java:156)
    at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:105)
    at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:53)
    at com.streamsets.datacollector.runner.Pipeline.initPipe(Pipeline.java:299)
    at com.streamsets.datacollector.runner.Pipeline.init(Pipeline.java:214)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:96)
    at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:79)
    at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:646)
    at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:143)
    at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:233)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang ...
(more)
edit retag flag offensive close merge delete

Comments

Can you post the full stack trace from sdc.log, including all the 'caused by' traces?

metadaddy gravatar imagemetadaddy ( 2017-09-22 11:57:45 -0500 )edit

I am having the same error, did you get it to work?

Kailegh gravatar imageKailegh ( 2018-05-14 09:12:56 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2017-10-03 10:10:00 -0500

metadaddy gravatar image

It looks like you're missing some of the Hive jars (evidenced by the java.lang.NoClassDefFoundError in the log). You need the following jars (as listed in the Hive docs):

  • hive-jdbc*.jar
  • hive-service*.jar
  • libfb303-0.9.0.jar
  • libthrift-0.9.0.jar
  • log4j-1.2.16.jar
  • slf4j-api-1.6.1.jar
  • slf4j-log4j12-1.6.1.jar
  • commons-logging-1.0.4.jar

You should use the Package Manager to install these jars for use by the JDBC stage library, or drop them in ${STREAMSETS_LIBRARIES_EXTRA_DIR}/streamsets-datacollector-jdbc-lib/lib/ manually.

edit flag offensive delete link more

Comments

Hi, Thanks a lot for you answer. I will have the jars installed so I can test. I will let you know how it went after I have the jar files installed. Thanks again

hdelgad gravatar imagehdelgad ( 2017-10-03 10:33:08 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-09-21 21:45:05 -0500

Seen: 334 times

Last updated: Oct 03 '17