Listing Jobs via Jobs API
Instructions on using Jobs API to list previously created jobs
This page provides instructions on how to list jobs using the Jobs API. The API allows you to retrieve information about jobs associated with a specific pipeline and component within your Datastreamer dynamic-pipelines environment.
Required Parameters
Before using the Jobs API, ensure you have the following:
- Pipeline ID: The ID of the pipeline for which you want to list jobs.
- Component ID: The ID of the component within the pipeline.
GET Jobs
To list jobs, you can use the following endpoint. Make sure to replace the placeholders with your actual values
curl --location 'https://api.platform.datastreamer.io/api/pipelines/:PipelineId/components/:ComponentId/jobs?count=50' \
--header 'apikey: API_KEY'
Example
Here is an example of how the curl command might look after replacing the placeholders:
curl --location 'https://api.platform.datastreamer.io/api/pipelines/12345/components/67890/jobs?count=50' \
--header 'apikey: abcdef123456'
The API will return a JSON response containing details of the jobs, such as job IDs, statuses, and timestamps.
{
"jobs": [
{
"organization_id": "129706a7-0af1-4dc5-8eae-06404a89c3ca",
"pipeline_id": "data365-twitter-keywords-1",
"step_id": "docsin0",
"job_id": "7c4623a3-10da-4638-b90f-995b7ca6d20e",
"job_name": "test_twitter_keywords",
"work_item_id": "aa7ec53c-b1df-43c2-b403-785e4ceb3c4a",
"data_source": "twitter_keywords",
"job_type": "oneTime",
"job_state": "succeeded",
"label": null,
"tags": null,
"created": "2024-05-01T21:23:36.627326Z",
"updated": "2024-05-01T21:23:36.627326Z",
"preferred_start_time": "2024-05-01T21:27:38.945018Z",
"query_from": "2024-01-01T00:00:00Z",
"query_to": "2024-02-01T00:00:00Z",
"query": "raspberry pi",
"max_documents": 0,
"work_time_offset": 0,
"start_time_offset": 0
},
{
"organization_id": "129706a7-0af1-4dc5-8eae-06404a89c3ca",
"pipeline_id": "data365-twitter-keywords-1",
"step_id": "docsin0",
"job_id": "a4a7f219-1eaf-42ff-bfb4-5a637d80fc44",
"job_name": "test_twitter_keywords",
"work_item_id": "c08aa1ab-3f7b-465b-9a68-f9cbb01d3697",
"data_source": "twitter_keywords",
"job_type": "oneTime",
"job_state": "running",
"label": null,
"tags": null,
"created": "2024-05-01T22:17:05.653031Z",
"updated": "2024-05-01T22:17:05.653031Z",
"preferred_start_time": "2024-05-02T16:08:22.335522Z",
"query_from": "2024-01-01T00:00:00Z",
"query_to": "2024-02-01T00:00:00Z",
"query": "raspberry pi",
"max_documents": 0,
"work_time_offset": 0,
"start_time_offset": 0
},
{
"organization_id": "129706a7-0af1-4dc5-8eae-06404a89c3ca",
"pipeline_id": "data365-twitter-keywords-2",
"step_id": "docsin0",
"job_id": "082109a9-32a7-4bfe-946d-e91b22a4eb2c",
"job_name": "test_twitter_keywords",
"work_item_id": "c39e24e5-1fbd-4aee-a789-9c996b6b0e5f",
"data_source": "twitter_keywords",
"job_type": "oneTime",
"job_state": "failed",
"label": null,
"tags": null,
"created": "2024-05-02T19:20:52.310094Z",
"updated": "2024-05-02T19:20:52.310094Z",
"preferred_start_time": "2024-05-02T19:21:19.140537Z",
"query_from": "2024-01-01T00:00:00Z",
"query_to": "2024-02-01T00:00:00Z",
"query": "dot matrix",
"max_documents": 0,
"work_time_offset": 0,
"start_time_offset": 0
}
],
"page": 1,
"page_count": 50,
"total_count": 3
}
The count parameter specifies the number of jobs to return. Adjust this number based on your needs.
Updated about 2 months ago