Ask Your Question

Json schema validation

asked 2018-07-09 14:47:11 -0500

KRT gravatar image

Can you please let me know how to validate the incoming JSON data in streamsets?

For example, I am expecting the incoming JSON only as below format. If any key name is missed I need to send that record to error pipeline

{ "id":12345, "name":"ABC", "address":"NewYork" }

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-07-09 15:49:49 -0500

iamontheinet gravatar image

Hi Karthy,

Assuming you're using one of the processors downstream, set 'Required Fields' as documented here in configuration section of any of the processors.

Cheers, Dash

edit flag offensive delete link more


Hi Dash, Thanks for your response :) I need to validate the datatype as well. In my example, I have numeric and string datatype and the incoming data should satisfy those as well

KRT gravatar imageKRT ( 2018-07-10 08:38:00 -0500 )edit

You're welcome! Within the same Jython or JavaScript processor (that checks for required fields) you can write your code in 'Script' section to check for datatypes.

iamontheinet gravatar imageiamontheinet ( 2018-07-10 11:17:30 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-07-09 14:47:11 -0500

Seen: 346 times

Last updated: Jul 09 '18