Ask Your Question
0

Pipeline fails with OutOfMemory "Java Heap size" exception after continuous mysql binlog issue

asked 2020-10-05 06:26:13 -0500

abdulsyed gravatar image

My pipeline is about replicating MySQL Binlog to s3, but when the pipeline runs continuously, It stops and goes into the below state "Run_ERROR": "Java Heap Size"

Exception: java.lang.OutOfMemoryError: Java heap space java.lang.OutOfMemoryError: Java heap space Exception

I'm running streamsets-dc v3.16 on docker Ec2 r3.xlarge - 16GB Memory (8 GB allocated for Heap space).

Also the pipeline continuously gets restarted with "MySql server error: Client requested master to start replication from position > file size"

Can anyone help me with this issue?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2020-10-05 22:27:10 -0500

iamontheinet gravatar image

Hi!

Here's a great explanation on what is going on and how you might be able to resolve the issue -- https://misfra.me/2017/08/02/recoveri...

Cheers, Dash

edit flag offensive delete link more

Comments

1

Thanks for the response. for the second part of my question, "Java HeapSize Error" Is this also why the heap size error is caused?

abdulsyed gravatar imageabdulsyed ( 2020-10-05 23:50:27 -0500 )edit

It appears that the two are related.

iamontheinet gravatar imageiamontheinet ( 2020-10-06 00:16:44 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-10-05 06:26:13 -0500

Seen: 129 times

Last updated: Oct 05