Ask Your Question
0

Copy/Re-use Stages between Pipelines

asked 2019-07-19 15:38:11 -0500

DataAnalyst1029 gravatar image

updated 2019-07-19 15:38:39 -0500

I have multiple pipelines that I am making regular changes to, and require a way to quickly copy a stage, configurations and all, into another pipeline. Is there a specific way to do this? I see that there is a Sharing ability in pipelines, but am not sure from documentation whether this is for my specific issue.

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
0

answered 2019-07-19 17:43:36 -0500

iamontheinet gravatar image

Hi!

Not sure if you're aware of duplicating pipelines. The other option is pipeline fragments, available via StreamSets Control Hub.

Cheers, Dash

edit flag offensive delete link more

Comments

The pipelines are already configured, so I would have to transfer existing processing logic over to any duplicated pipeline. The pipeline fragments approach does look promising, I will look into this!

DataAnalyst1029 gravatar imageDataAnalyst1029 ( 2019-07-20 11:09:10 -0500 )edit
1

answered 2019-09-04 14:23:20 -0500

Carol gravatar image

Typically when I have to copy a Stage (configurations and all) I use 'copy' and 'paste' and it works fine. The only gotcha with this approach is that the copy/paste feature does not work in Control Hub. It works if the pipeline you are editing is on Data Collector. So I typically download form the repository to a data collector and work there, if i need to copy a bunch of stages quickly.

edit flag offensive delete link more

Comments

Thanks Carol! I actually tried this and realized that the original browser I was using (an older version of Firefox) was not allowing me to copy/paste correctly between tabs. Attempting this in Chrome worked perfectly!

DataAnalyst1029 gravatar imageDataAnalyst1029 ( 2019-09-04 15:52:19 -0500 )edit

Glad it worked fine. I see now that I forgot to mention I use Chrome, sorry about that. Might have saved you some effort.

Carol gravatar imageCarol ( 2019-09-04 15:55:06 -0500 )edit
1

Definitely not your fault - I should have tried multiple browsers to begin with. Thanks for the answer!

DataAnalyst1029 gravatar imageDataAnalyst1029 ( 2019-09-04 16:01:52 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-07-19 15:38:11 -0500

Seen: 101 times

Last updated: Sep 04