YouTube

The YouTube source collects videos from YouTube as part of a Data Stream.


📘

Datastreamer selects from available providers automatically based on your Job configuration. You do not need to manage provider accounts or credentials to use this source.

API data_source: youtube_auto

Configuring a Job

The query type decides how a YouTube Job searches: across the whole platform or from one account. Each Job uses one query type, set in query.query_type (the Query Type field on the Portal's New Job form), with query.platform set to youtube. To cover more than one, create a Job for each.

  • Search by topic. keyword finds posts that mention your terms anywhere on YouTube.
  • Follow an account. user collects posts from one account.
  • Full control. advanced takes a Lucene query, so you can combine terms and search specific fields. See Advanced Search Query.

Query types

Query Type (Portal)query_typeFieldsNotes
Posts by keywordkeywordqueryKeyword to search for in videos.
Posts by useruserqueryThe YouTube user ID. Example: starbucks.
Advanced Search QueryadvancedqueryLucene query over unified schema fields. See Advanced Search Query.

The query object for each type:

{"platform": "youtube", "query_type": "keyword", "query": "electric vehicles"}
{"platform": "youtube", "query_type": "user", "query": "starbucks"}
{"platform": "youtube", "query_type": "advanced", "query": "(electric OR hybrid) AND vehicle"}

Job options

FieldDescription
query.prioritizationfreshness favors newer documents; coverage favors collecting a greater volume. See Freshness or Coverage.
job_typeonetime runs once over a date range; periodic runs on a schedule.
query_from, query_toOne-time Jobs only. Start and end of the date range, in ISO 8601 format. Both are required.
max_documentsOptional. Maximum documents to return. This is a soft limit: because pipelines are asynchronous, a few more documents may be returned.
label, tagsOptional. A label for grouping Jobs, and tags for filtering.

Advanced Search Query

The advanced query type takes a Lucene query in query.query. You can search any field in the Datastreamer unified schema; a term without a field searches content.body.

{ "platform": "youtube", "query_type": "advanced", "query": "(\"electric vehicle\" OR ev) AND content.hashtags:charging" }

Do not put dates in the query. Set the date range with query_from and query_to on the Job instead.

See the Schema Reference for field names.

Example Job request

curl --request POST \
  --url 'https://api.platform.datastreamer.io/api/pipelines/{PIPELINE_ID}/components/{COMPONENT_ID}/jobs?ready=true' \
  --header 'apikey: <your-api-key>' \
  --header 'Content-Type: application/json' \
  --data '{
  "job_name": "youtube-example",
  "component_name": "youtube-auto",
  "query": {
    "platform": "youtube",
    "query_type": "keyword",
    "query": "electric vehicles",
    "prioritization": "coverage"
  },
  "data_source": "youtube_auto",
  "job_type": "onetime",
  "query_from": "2026-10-01T00:00:00Z",
  "query_to": "2026-10-02T00:00:00Z",
  "max_documents": 1000
}'

Refer to the Job creation documentation for full configuration details: Creating Jobs

API Reference: Create a Job


What is Collected

Each document returned from the YouTube source represents a video. Fields are mapped to the Datastreamer unified schema and include content title and description, author metadata, engagement metrics (views, likes, comments), and publish date.

For audio and video transcription from YouTube content, see Direct Integration for enrichments to apply.


Troubleshooting

Job fails or returns no data

  • Check that the query or channel target is valid and publicly accessible
  • Verify the date range contains data
  • Review Job logs for specific errors

Unexpectedly high document counts

  • Scope the query to specific channels rather than broad keywords
  • Set a document limit on the Job
  • Review the DVU pricing page

Provider switch noted in Job logs

  • This is expected behavior. If a provider is unavailable, the Job is routed to an alternative automatically. No action is required.

Related


Did this page help you?