Ask Your Question

Avro schema validation failed

asked 2018-07-03 06:29:14 -0500

kutny gravatar image

I'm not able to set up Avro validation correctly. Any idea what is wrong?


avro schema

Is it somehow possible to select Avro validation schema based on expression (e.g. table name)? Right now I'm only able to set one Schema ID per pipeline which is not very practical.


edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-07-03 14:50:51 -0500

metadaddy gravatar image

Your schema is saying that the output record should have two fields in its root, id and name; I can see that the record contains /BinLogFilename, /Type, /Table etc but not /id and /name - I suspect they're somewhere in /Data. If you manipulate the record to get the right data into those fields (Expression Evaluator or Field Renamer are useful for this), the destination should be able to write them out.

You can't configure multiple schema ID's per pipeline (that would be an interesting enhancement, to allow expressions in the Schema ID field, if you'd like to suggest it at, but you can specify the Avro schema in the record header. You could configure a Static Lookup processor to provide the correct schema based on the value of some field.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools



Asked: 2018-07-03 06:29:14 -0500

Seen: 433 times

Last updated: Jul 03 '18