Ask Your Question

Column headers and values in separate arrays

asked 2020-02-02 11:53:20 -0600

pow gravatar image

Hi all,

trying out the Streamsets Cloud to pull report data from an API. In the returned JSON there is an array of objects that are the columns within the report. The position of the columns is important. After the columns is what is technically an array of arrays without any keys. Each "sub"-array represents a line in the report and is a list of values whose position on the list within each "sub"-array sequentially corresponds to the column. For example, if the second "column" is "last name", the second "value" in each "sub"-array is the value.

Here is a sample:

  "reports": [{
    "columns": [{
      "name": "profiles_first_name",
      "type": "text",
      "label": "First name"
    }, {
      "name": "profiles_last_name",
      "type": "text",
      "label": "Last name"
    }, {
      "name": "profiles_email",
      "type": "text",
      "label": "Company email"
    }, {
      "name": "profiles_guid",
      "type": "text",
      "label": "GUID"
    "content": [
      ["Test", "Profile", "", "3cd80406-708b-451a-ae31"],
      ["Testy", "Testfact", "", "c188b8ee-2877-4e88-b764"]

I want to get this back into a simple tabular report with column labels as headers and content as lines. I've been trying with the various field processors but can't seem to figure out how to get there. Anyone tried something like this before?

Thank you for your help in advance,

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2020-02-03 11:52:33 -0600

metadaddy gravatar image

I don't believe this is currently possible with StreamSets Cloud. In StreamSets Data Collector, you could do this with one of the script evaluators, but they are not yet available in Cloud.

edit flag offensive delete link more


Thank you very much

pow gravatar imagepow ( 2020-02-05 01:56:21 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2020-02-02 11:53:20 -0600

Seen: 70 times

Last updated: Feb 03