Does a local SDC Cron origin support more than one pipeline at a time?

asked 2020-04-29 15:16:11 -0500

oberfirer gravatar image

updated 2020-04-29 15:36:20 -0500

I have 2 pipelines, each starting with a "Cron" origin on different schedules: 0 0/10 * 1/1 * ? * and 0 0/15 * 1/1 * ? * (every 10 and every 15 minutes). I can run one at a time without issues, but when I try to start up the second one if one is already running, it seems like the first one stops. The first pipeline is still shown in the running state, but the timer does not kick off the flow at the expected intervals. Is that a bug or a limitation of a local SDC environment? Are there workarounds available? I tried it on 2 different versions of the SDC - 3.13.0 and 3.14.0. Also, when the second pipeline starts, I get a brief message at the top of the pipeline saying it "finished", immediately after it starts, and I have to click the start button again to really start it. I see an error in the logs:

org.quartz.ObjectAlreadyExistsException: Unable to store Job : 'dataCollectorJobGroup.dataCollectorJob', because one already exists with this identification. at org.quartz.simpl.RAMJobStore.storeJob(RAMJobStore.java:279) at org.quartz.simpl.RAMJobStore.storeJobAndTrigger(RAMJobStore.java:251) at org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:855) at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249) at com.streamsets.pipeline.stage.origin.scheduler.SchedulerPus

P.S. I may have found this was already submitted as a bug and resolved here: https://issues.streamsets.com/browse/... Please let me know when the fix is going to be available in a nightly or stable build.

Thank you, Oleg

edit retag flag offensive close merge delete