Ask Your Question
1

How to filter BOM in HTTP Client GET Request

asked 2020-03-03 10:36:56 -0500

avdsa gravatar image

updated 2020-03-03 13:13:32 -0500

I have been running into an error when trying to pass a GET request to the following api:

https://financialmodelingprep.com/api/v3/company/stock/list

The error returned is as follows:

Cannot parse record: com.fasterxml.jackson.core.JsonParseException: JSON array expected but stream starts with 'START_OBJECT'

When researching workarounds, I found the following StreamSets issue SDC-11039

It appears the issue with the the UTF-8 encoding and the API using a BOM. Is there a way for me to filter this in the request handler? Or do I have to wait for the enhancement to be finished?

Thank you.

Here is my origin setup: image description

image description

image description

image description

image description

image description

image description

edit retag flag offensive close merge delete

Comments

When I GET that URL, I get JSON that Data Collector can parse without a problem. Are you using some option to get XML?

metadaddy gravatar imagemetadaddy ( 2020-03-03 12:09:28 -0500 )edit

I do not believe so. I updated my question with the breakdown of my origin parameters

avdsa gravatar imageavdsa ( 2020-03-03 13:13:53 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2020-03-03 16:54:17 -0500

metadaddy gravatar image

You are specifying 'JSON array of objects', but that API actually returns a single JSON object that itself contains an array:

{
  "symbolsList" : [ {
    "symbol" : "SPY",
    "name" : "SPDR S&P 500",
    "price" : 300.24,
    "exchange" : "NYSE Arca"
  },
  ...
  ]
}

You should change the JSON Content setting to 'Multiple JSON objects'. Your origin will emit one big record with all the data from the API call. You can use Field Pivoter to break this out into separate records:

image description

The result will look like this:

image description

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-03-03 10:36:56 -0500

Seen: 43 times

Last updated: Mar 03