📘

Pipeline Required

If you do not yet have a Pipeline created, you should create a pipeline first. The Pipeline ID and Step ID will be required for adding Jobs to the correct Pipeline and step.

🚧

Only Periodic Jobs can be updated

If you wish to manage Jobs via API, you will need to setup the Pipeline initially in the Portal UI. After setting up the Pipeline, you can create new Jobs through APIs.

Updating a Job within Portal

Within Portal, navigate to your Pipeline. If the Pipeline is not running, you will need to select "Edit".

Then, you can select the Component that you wish to edit Jobs for, and it will open a configuration Sidebar. You will be able to edit Scheduled Jobs.

After entering changes, press "Save" and the new configuration will be set. You can do the same from the Pipeline Jobs Page.

Updating a Job Via API

API calls to update Jobs are interacting with this Pipeline Component; so it is required to have both correct Pipeline ID and Step ID.

Follow the Portal steps above to the Ingress Component "Manage Jobs" view to retrieve the Step ID and Pipeline ID which form part of the request URL.

📘

Quick Tip

Updating a job uses the same JSON structure as job creation, therefore you can use the code button in the UI page to see the format of the job for each data source.

Job Update API Request

Here is a sample Job Update API Request.

curl --request PUT --location 'https://api.platform.datastreamer.io/api/pipelines/:PipelineId/components/:ComponentId/jobs/:JobId' \
			--header 'Content-Type: application/json' \
			--header 'apikey: API_KEY' \
			'{
   				... job JSON payload
			}'

Example Request

curl --request PUT --location 'https://api.platform.datastreamer.io/api/pipelines/12345/components/67890/jobs/job123 \
      --header 'apikey: abcdef123456' \
      --header 'Content-Type: application/json' \
      --data \
        '{
          "job_name": "bf5912e9-5e09-45c8-9a61-3ea75fbc7248",
          "data_source": "socialgist_blogs",
          "query": {
            "query": "cats AND dogs",
            "country": "FR",
            "language": "fr"
          },
          "job_type": "periodic",
          "schedule": "0 0 0/6 1/1 * ? *",
          "label": "Test Label"
        }'