Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

jdbc.tables in JDBC Multitable Consumer

Hi,

There are two tables in the Netezza database that I want to write the two tables as individual CSV files to hdfs. Below are the steps I follow (described in detail) but I still just see a single CSV file created with records from both tables in this single file and I do not see the record header attribute jdbc.tables written to the file. How should I use the jdbc.tables in the local FS destination to stream records to different files.

1) JDBC Multitable Consumer:

  • Set the JDBC Connection String
  • Per Batch Strategy = Process All available rows from the table
  • Set my Schema name under Table config
  • Set Table Name Pattern as ABC%
  • Enable Non-Incremental Load

2) Expression Evaluator:

  • Under Expressions, set Header Attribute as <jdbc.tables>
  • Set Header Attribute Expression as ${record:attribute('jdbc.tables')}

3) Local FS:

  • Set File Type = Text Files
  • Set Files Prefix = sdc-${sdc:id()}
  • Set Directory Template = /mapr/dsrx/data/ods/ph_contract/Test1/${YYYY()}-${MM()}-${DD()}-${hh()}
  • Set Data Format = Delimited CSV (ignore empty lines)
  • Set Header line = With Header line
  • Enable Replace New Line Characters

Thanks a lot.