Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

We can start the pipeline using the Curl commands, As always we can schedule the Curl job from Cron job or oozie or any other workflows

Below is the simple script that can initiate the curl command to start the stream sets pipeline, hope it helps someone !!

Shell script to run the Stream sets pipeline

We can start the pipeline using the Curl commands, As always we can schedule the Curl job from Cron job or oozie or any other workflows

Below is the simple script that can initiate the curl command to start the stream sets pipeline, hope it helps someone !!

Shell script to run the Stream sets pipeline

#!bin/bash
#Shell Script to run the pipeline using Curl
 #Hostname Of Stream sets
SSHost=${1}

#Streamsets UI port
SSPort=${2}

#Streamsets pipeline ID
SSPipeID=${3}

#Stream sets Login User
SSUser=${4}

#Stream sets password
SSPW=${5}

echo "SS Host            : "${SSHost}
echo "SS Port            : "${SSPort}
echo "SS Pipe ID         : "${SSPipeID}
echo "SS User            : "${SSUser}

#Using curl to start the Stream sets pipeline
curl -H "X-Requested-By:sdc" -X POST  http://${SSHost}:${SSPort}/rest/v1/pipeline/${SSPipeID}/start  -u 
${SSUser}:${SSPW}

We can start the pipeline using the Curl commands, As always we can schedule the Curl job from Cron job or oozie or any other workflows

Below is the simple script that can initiate the curl command to start the stream sets pipeline, hope it helps someone !!

#!bin/bash
#Shell Script to run the pipeline using Curl

#Hostname Of Stream sets
SSHost=${1}

#Streamsets UI port
SSPort=${2}

#Streamsets pipeline ID
SSPipeID=${3}

#Stream sets Login User
SSUser=${4}

#Stream sets password
SSPW=${5}

echo "SS Host            : "${SSHost}
echo "SS Port            : "${SSPort}
echo "SS Pipe ID         : "${SSPipeID}
echo "SS User            : "${SSUser}

#Using curl to start the Stream sets pipeline
curl -H "X-Requested-By:sdc" -X POST \
    http://${SSHost}:${SSPort}/rest/v1/pipeline/${SSPipeID}/start  -u  ${SSUser}:${SSPW}