Stop pipeline when MapReduce Executor ends

asked 2018-11-07 04:17:43 -0600

elisavet gravatar image


My goal is to send data from sql to hive in parquet format as described here

The problem is that although the pipeline finishes, MapReduce continues to transform avro files to parquet.

So, I was wondering, is there a way to stop the pipeline only when all MapReduce jobs have finished?

Thank you, Elisavet

edit retag flag offensive close merge delete