Ask Your Question

How to build and deploy Streamsets pipeline into different Environments through GIT

asked 2019-02-12 05:32:39 -0500

Vejella gravatar image

updated 2019-02-14 03:47:34 -0500

I am new to Stream sets and I need steps to Build and deploy stream sets pipelines from DEV environment to UAT/PRD env's . Please let me know the steps to Deploy pipelines.

Please explain the following steps :

• How to add the stream sets pipeline stages code in GIT .

• What are the files I need to add in GIT

• How to Build and deploy the Stream sets pipeline

• How o move the stream sets pipeline code from DEV Environment to UAT/PROD Environments through GIT and steps to perform while running the stream sets in UAT/PROD env's

edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted

answered 2019-02-13 14:23:54 -0500

trajan gravatar image

Streamsets provides list of API to work with the pipeline, including import a new pipeline, start a pipeline etc. using this API you should be able to pull the json file from GIT and import it using the API. you can get the list of API's by clicking on the help button at the top right corner and select RESTful API

edit flag offensive delete link more


Thanks Rajan , Can you please provide me the steps in detail as I am new to this tool . My question is if I create new Pipeline with stages in DEV env and I want to build and deploy it in Different env i.e say DEV to QA/PROD then what are the steps I need to perform.How add my pipeline code in GIT

Vejella gravatar imageVejella ( 2019-02-14 03:56:16 -0500 )edit

I'm new to streamsets too. You can always use contorl-hub for managing the pipeline. Other than that, this is what i think is feasible. Export the pipeline as json file and store them in github. You can use tool like jenkins or similar take the json file and use the REST Api to import json

trajan gravatar imagetrajan ( 2019-02-15 11:29:55 -0500 )edit

Thank you Rajan for your suggestions. I will look into the Control Hub pipeline management and export/import options

Vejella gravatar imageVejella ( 2019-02-17 23:08:52 -0500 )edit
Login/Signup to Answer

Question Tools



Asked: 2019-02-12 05:32:39 -0500

Seen: 320 times

Last updated: Feb 14 '19