Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Create a csv file using Jython Evaluator


I would like to use the Jython Evaluator to create a file with the CSV extension that will contain selected data from records from the pipeline. I'm using Jython Evaluator, and please find the code below:

import os
import csv
from datetime import datetime

for record in records:
    srcdir = record.value['srcDir']
    filename = record.value['metadataFileName']

    headers = ['repositoryName','docID','fileName']

    with open(os.path.join(srcdir, filename + ".csv"), 'w') as csvfile:
      writer = csv.writer(csvfile, dialect = 'excel', delimiter=',')
      for record in records:
        writer.writerow([record.value['repositoryName'], record.value['docID'], record.value['fileName']])

#writing to the pipeline output

  except Exception as e:
    error.write(record, str(e))

The csv file is being saved in the correct format:

image description

There is only one issue. For example, I have three records in one batch, one of them has a creation date: May 1, 2020 and the other two are from May 5, 2020. The record from 1st comes to Documents_2020-05-01 directory, from 5th to Documents_2020-05-05. Each file has its own metadata.csv file. At the moment each metadata file has all three records/rows. I want to group them.

There are three records/current view: image description

Required final view:

metadata1.csv: (in Documents_2020-05-01 folder)

image description

metadata2.csv: (in Documents_2020-05-05 folder)

image description