Hello : I use kudu as the destination of my pipeline 。But occationally in the pipeline's log it reads that
"Errors while interacting with Kudu: org.apache.kudu.client.NonRecoverableException: MANUAL_FLUSH is enabled but the buffer is too big" 。
How should I figure out the problem 。
Finally I get the answer . while interacting with Kudu ,kudu client has a default config----"Mutation Buffer Space " which is 1000.this config control the size of one flush of kudu。 At ordinary times,the pipeline runs smoothly,But when DBA operate the databases Manully,delete many rows accordingly to some conditions once. If the number of deleted rows exceed the config of "Mutation Buffer Space " .then the above issue occurs.
Asked: 2019-07-09 20:17:58 -0500
Seen: 307 times
Last updated: Jul 10 '19
