Ask Your Question
1

how do I drop multiple hive tables before writing to it

asked 2019-11-14 08:54:51 -0500

Ranjan Mishra gravatar image

I have to load multiple s3 CSV to hive. The file name, contents keep on changing in every load so I have to drop the existing tables in HIVE, create and full refresh. In the same hive database there are other tables but I can use wild card as the tables for this particular load can have a fixed prefix such as APP10_

how do I do it

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
1

answered 2019-12-09 05:58:38 -0500

Ranjan Mishra gravatar image

updated 2019-12-09 05:59:08 -0500

Thank you Gangi My table names and list could possibly change so I need to generate the drop statements dynamically. I can do this in shell executor by generating a shell script dynamically and calling it from start event - shell exector

edit flag offensive delete link more
1

answered 2019-12-06 08:51:48 -0500

yg gravatar image

The same can be achieved by enabling the hive Query as start event for the pipeline and specify the drop statement in start event-Hive Query Configuration Tab as below.

Steps:

General Tab of pipeline--> select Hive Query as start event --> this will opens a tab start event-Hive Query.

In start event-Hive Query configure required details like hive jdbs Url, query that you want run(drop table if exists <hiveschema>.<tablename>)

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-11-14 08:54:51 -0500

Seen: 185 times

Last updated: Dec 09 '19