Ask Your Question

sdc kudu destination buffer too big

asked 2019-07-09 20:17:58 -0500

lichanghong gravatar image

updated 2019-07-09 20:18:52 -0500

Hello : I use kudu as the destination of my pipeline 。But occationally in the pipeline's log it reads that

"Errors while interacting with Kudu: org.apache.kudu.client.NonRecoverableException: MANUAL_FLUSH is enabled but the buffer is too big" 。

How should I figure out the problem 。

edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted

answered 2019-07-10 21:17:28 -0500

lichanghong gravatar image

Finally I get the answer . while interacting with Kudu ,kudu client has a default config----"Mutation Buffer Space " which is 1000.this config control the size of one flush of kudu。 At ordinary times,the pipeline runs smoothly,But when DBA operate the databases Manully,delete many rows accordingly to some conditions once. If the number of deleted rows exceed the config of "Mutation Buffer Space " .then the above issue occurs.

edit flag offensive delete link more


Thanks for coming back and posting the solution!

metadaddy gravatar imagemetadaddy ( 2019-07-10 21:24:52 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2019-07-09 20:17:58 -0500

Seen: 307 times

Last updated: Jul 10 '19