Ask Your Question

HIVE_30 - Invalid column name 

asked 2020-03-19 11:24:16 -0500

Carol gravatar image

I have created a pipeline that reads text files (extension of .dat) using an SFTP Origin. I can preview the pipeline and read the file contents.

I need to write the file contents to Hive, and so I use the HIVE METADATA processor to configure what is required. The Table Name in the HIVE MEtADATA configuration is a parameter I defined because i want all the files read in to write to the same hive table.

For some strange reason, i receive an error : HIVE_30 - Invalid column name error and it names the first column/field in the input file. The files are tab delimited.

Any pointers on where else i can look/check ?

edit retag flag offensive close merge delete


I noticed one unusual thing. In the pipeline Preview mode, the column on which i am getting the 'invalid column name' error appears to have its name surrounded by single quotes, unlike all the others. Odd. Source file has no quotes around the names.

Carol gravatar imageCarol ( 2020-03-19 13:41:31 -0500 )edit

1 Answer

Sort by » oldest newest most voted

answered 2020-03-23 08:22:14 -0500

Carol gravatar image

Issue resolved. Thanks to a colleague who helped figure this out.

Turns out the first column name in the flat text file has a strange imbedded character in the first position.
A 'Field Renamer' processor did the trick.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2020-03-19 11:24:16 -0500

Seen: 59 times

Last updated: Mar 23 '20