Create a Job

post

https://api.platform.datastreamer.io/api/pipelines/{pipelineId}/components/{componentId}/jobs

Creates a Job on the Ingress Component.

Recent Requests

Time	Status	User Agent
Retrieving recent requests…

Loading…

Path Params

pipelineId

string

required

The pipeline ID.

componentId

string

required

The Ingress Component (step) ID.

Query Params

ready

boolean

Set to true to create Jobs ready to run. Without it, Jobs are created in the NotReady state and stay inactive until started with the state update endpoint.

Body Params

job_name

string

required

Identifies the Job in analytics and management screens. Any string.

data_source

string

required

The data source, for example twitter_auto.

query

object

required

The source query. Its shape depends on the data_source. Auto source Jobs use { "query": "" }.

job_type

string

enum

Defaults to onetime

onetime runs once over query_from to query_to; periodic runs on schedule. Defaults to onetime if omitted. Responses return job_type as oneTime or periodic.

Allowed:

query_from

date-time

Start of the collection window, for one-time Jobs.

query_to

date-time

End of the collection window, for one-time Jobs.

schedule

string

For periodic Jobs: a 7-field cron expression, for example 0 0 0/6 1/1 ? . If it is more frequent than the source supports, the lowest supported interval is used.

max_documents

int32

A soft cap on documents per Job run. The final count can be slightly higher.

label

string

A single string applied to every document the Job collects.

200Success