Listing Jobs via Jobs API

This page provides instructions on how to list jobs using the Jobs API. The API allows you to retrieve information about jobs associated with a specific pipeline and Ingress Component within your Datastreamer dynamic-pipelines environment.

To search across pipelines without those IDs, use Advanced Job Search and filter on a known field such as state or job_id. The Jobs API currently has no unfiltered, organization-wide GET /api/jobs endpoint.

Required Parameters

Before using the Jobs API, ensure you have the following:

Pipeline ID: The ID of the pipeline for which you want to list jobs.
Component ID: The ID of the component within the pipeline.

GET Jobs

To list jobs, you can use the following endpoint. Make sure to replace the placeholders with your actual values

curl --location 'https://api.platform.datastreamer.io/api/pipelines/:PipelineId/components/:ComponentId/jobs?count=50' \
--header 'apikey: API_KEY'

API Reference: List Jobs

Example

Here is an example of how the curl command might look after replacing the placeholders:

curl --location 'https://api.platform.datastreamer.io/api/pipelines/12345/components/67890/jobs?count=50' \
--header 'apikey: abcdef123456'

The API will return a JSON response containing details of the jobs, such as job IDs, statuses, and timestamps.

{
    "start": 0,
    "count": 1,
    "records": [
        {
            "type": "job",
            "organization_id": "11111111-1111-1111-1111-111111111111",
            "pipeline_id": "12345",
            "step_id": "67890",
            "job_id": "job123",
            "state": "succeeded",
            "job_name": "test_twitter_auto",
            "data_source": "twitter_auto",
            "job_type": "oneTime",
            "priority": "normal",
            "preferred_start_time": "2026-09-15T19:15:31.910854Z",
            "query_from": "2026-09-14T00:00:00Z",
            "query_to": "2026-09-15T00:00:00Z",
            "query": {
                "platform": "twitter",
                "query": "test",
                "prioritization": "coverage",
                "per_day_limit": null,
                "include": [ { "max_documents": null } ]
            },
            "query_start_offset": 0,
            "start_time_offset": 0,
            "max_documents": 10,
            "secondary_max_documents": 0,
            "created": "2026-09-15T19:15:28.60794Z",
            "updated": "2026-09-15T19:16:32.420828Z",
            "started": "2026-09-15T19:15:31.919687Z",
            "ended": "2026-09-15T19:16:32.420828Z",
            "schedule_time": "2026-09-15T19:15:31.919687Z",
            "document_count": 10,
            "secondary_document_count": 0,
            "elapsed_time_ms": 2899,
            "run_count": 1,
            "error": ""
        }
    ]
}

The jobs are returned under records. Each record uses the same shape as the Get Job Details response, including state, the object-form query, and the timing fields.

📘
The count query parameter sets how many jobs to return, and start sets the position of the first job to return. For example, ?count=50&start=50 returns the second page of 50. The response echoes start and count and returns the jobs under records.

The response count is the number of records returned. Add ?total=true to include the total number of matching jobs; without it, total is omitted.

Required Parameters

GET Jobs

Example

The count query parameter sets how many jobs to return, and start sets the position of the first job to return. For example, ?count=50&start=50 returns the second page of 50. The response echoes start and count and returns the jobs under records.

The `count` query parameter sets how many jobs to return, and `start` sets the position of the first job to return. For example, `?count=50&start=50` returns the second page of 50. The response echoes `start` and `count` and returns the jobs under `records`.