Hello : I use kudu as the destination of my pipeline 。But occationally in the pipeline's log it reads that
"Errors while interacting with Kudu: org.apache.kudu.client.NonRecoverableException: MANUAL_FLUSH is enabled but the buffer is too big" 。
How should I figure out the problem 。
Finally I get the answer . while interacting with Kudu ,kudu client has a default config----"Mutation Buffer Space " which is 1000.this config control the size of one flush of kudu。 At ordinary times,the pipeline runs smoothly,But when DBA operate the databases Manully,delete many rows accordingly to some conditions once. If the number of deleted rows exceed the config of "Mutation Buffer Space " .then the above issue occurs.
@metadaddy, @lichanghong....We are also getting similar error while loading data to Kudu destination. Where do we need to change Mutation Buffer Space property ??
Asked: 2019-07-09 20:17:58 -0600
Seen: 491 times
Last updated: Jul 10 '19
