Ask Your Question

How can I use the StreamSets Control Hub API to see how many jobs are currently running in my cluster?

asked 2018-10-08 10:12:47 -0500

dcwatson84 gravatar image

updated 2018-10-08 14:53:12 -0500

metadaddy gravatar image

I have a set of jobs that are scheduled to run at different times, and at any given point in time I want to figure out exactly how many jobs are currently running, i.e. not just active, but actually running. What rest API call gives me the information to determine this?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-10-08 14:52:57 -0500

metadaddy gravatar image

You can use the /jobrunner/rest/v1/jobs/byStatus?jobStatus=ACTIVE REST API CALL to query for active jobs - pipeline status (RUNNING etc) is included in the response.

For example, I have one active job, with one running pipeline - you can see the RUNNING status in this response:

    "id": "3fd72a3f-cb97-4dc3-8a97-48af3705b6a4:metadaddy",
    "name": "Job for MySQL to Kafka",
    "description": null,
    "organization": "metadaddy",
    "creator": "admin@metadaddy",
    "createTime": 1538622464738,
    "lastModifiedBy": "admin@metadaddy",
    "lastModifiedOn": 1539027058225,
    "destroyer": null,
    "deleteTime": 0,
    "jobDeleted": false,
    "pipelineName": "MySQL to Kafka",
    "pipelineId": "e62a3b49-fc76-4c06-8075-4f59c67bd385:metadaddy",
    "pipelineCommitId": "0b23cd13-220c-471a-a95c-9508a4727a77:metadaddy",
    "rulesId": "accc8864-64cc-4e91-9f11-bafaae6d6261:metadaddy",
    "pipelineCommitLabel": "v4",
    "labels": [
    "currentJobStatus": {
      "id": "67b5ae89-53f4-4a42-b2ab-41e790c96fad:metadaddy",
      "jobId": "3fd72a3f-cb97-4dc3-8a97-48af3705b6a4:metadaddy",
      "pipelineCommitId": "0b23cd13-220c-471a-a95c-9508a4727a77:metadaddy",
      "runCount": 10,
      "user": "admin@metadaddy",
      "startTime": 1539027665879,
      "finishTime": 0,
      "errorMessage": null,
      "sdcIds": [
      "status": "ACTIVE",
      "warnings": [],
      "ackTrackers": [
          "sdcId": "4d979509-c754-11e8-b55d-db71045cb2fb",
          "createTime": 224556841797452,
          "ackStatus": "SUCCESS",
          "message": null,
          "eventType": "SAVE_PIPELINE"
          "sdcId": "4d979509-c754-11e8-b55d-db71045cb2fb",
          "createTime": 224556885212095,
          "ackStatus": "SUCCESS",
          "message": null,
          "eventType": "START_PIPELINE"
      "pipelineOffsets": null,
      "pipelineStatus": [
          "sdcId": "4d979509-c754-11e8-b55d-db71045cb2fb",
          "name": "MySQL to Kafka:3fd72a3f-cb97-4dc3-8a97-48af3705b6a4:metadaddy",
          "title": "MySQL to Kafka",
          "jobId": "3fd72a3f-cb97-4dc3-8a97-48af3705b6a4:metadaddy",
          "localPipeline": false,
          "status": "RUNNING",
          "lastReportedTime": 1539027893303,
          "worker": [],
          "message": null,
          "clusterMode": false
      "edge": false
    "parentJobId": null,
    "systemJobId": null,
    "numInstances": 1,
    "migrateOffsets": false,
    "statsRefreshInterval": 60000,
    "runtimeParameters": "{\"\":\"\"}",
    "edge": false,
    "timeSeries": false,
    "forceStopTimeout": 120000,
    "provenanceMetaData": {}
edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-10-08 10:12:47 -0500

Seen: 736 times

Last updated: Oct 08 '18