Ask Your Question

Revision history [back]

Hive metadata - partition shows twice

We have the following pipeline configuration on 2.5.0.0 ; image description

The table create statement : CREATE TABLE mytable (fields) partitioned by (MONTH_CAL_ID INT) stored as avro;

When the table is created, we only have one month_cal_id column.

After a first pipeline run, a second column month_cal_id shows up.

While this isn't an issue when querying with hive, spark throws an error : Reference 'MONTH_CAL_ID' is ambiguous, could be: MONTH_CAL_ID#69, MONTH_CAL_ID#70

We will update to check if this may already be fixed but for now Cloudera Manager doesn't shows 2.5.1.0 and 2.5.1.1 parcels. Have to do it manually.

Any suggestions ?

Hive metadata - partition shows twice

We have the following pipeline configuration on 2.5.0.0 2.5.1.1 ; image description

The table create statement : CREATE TABLE mytable (fields) partitioned by (MONTH_CAL_ID INT) stored as avro;

When the table is created, we only have one month_cal_id column.

After a first pipeline run, a second column month_cal_id shows up.

While this isn't an issue when querying with hive, spark throws an error : Reference 'MONTH_CAL_ID' is ambiguous, could be: MONTH_CAL_ID#69, MONTH_CAL_ID#70

We will update to check if this may already be fixed but for now Cloudera Manager doesn't shows 2.5.1.0 and 2.5.1.1 parcels. Have to do it manually.

Any suggestions ?