Ask Your Question

Pipeline fails with OutOfMemory "Java Heap size" exception after continuous mysql binlog issue

asked 2020-10-05 06:26:13 -0600

abdulsyed gravatar image

My pipeline is about replicating MySQL Binlog to s3, but when the pipeline runs continuously, It stops and goes into the below state "Run_ERROR": "Java Heap Size"

Exception: java.lang.OutOfMemoryError: Java heap space java.lang.OutOfMemoryError: Java heap space Exception

I'm running streamsets-dc v3.16 on docker Ec2 r3.xlarge - 16GB Memory (8 GB allocated for Heap space).

Also the pipeline continuously gets restarted with "MySql server error: Client requested master to start replication from position > file size"

Can anyone help me with this issue?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2020-10-05 22:27:10 -0600

iamontheinet gravatar image


Here's a great explanation on what is going on and how you might be able to resolve the issue --

Cheers, Dash

edit flag offensive delete link more



Thanks for the response. for the second part of my question, "Java HeapSize Error" Is this also why the heap size error is caused?

abdulsyed gravatar imageabdulsyed ( 2020-10-05 23:50:27 -0600 )edit

It appears that the two are related.

iamontheinet gravatar imageiamontheinet ( 2020-10-06 00:16:44 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2020-10-05 06:26:13 -0600

Seen: 189 times

Last updated: Oct 05 '20