Twitter/X
The Twitter/X source collects posts, replies, and public account content from Twitter/X as part of a Data Stream.
Datastreamer selects from available providers automatically based on your Job configuration. You do not need to manage provider accounts or credentials to use this source.
Configuring a Job
When creating a Job for the Twitter/X source, you define what content to collect. Common configuration options include:
- Query: a search expression using Twitter/X advanced search syntax (see below)
- Account targets: specific handles to monitor
- Date range: the time window to collect data from
- Language filter: restrict results to a specific language
- Document limit: maximum number of documents per Job run
Refer to the Job creation documentation for full configuration details: Creating Jobs
Do not include dates in your query string.
The Job system handles date ranges and historical searches. Including dates inside the query may cause errors or limit provider selection.
Twitter/X Query Syntax
Queries for the Twitter/X source use Twitter/X advanced search formatting. The table below covers the most common operators.
| Operator | Example | Description |
|---|---|---|
| Phrase | "dog house" | Exact phrase match |
| NOT | -mouse | Exclude this term |
| From | from:cat | Posts from this handle (no @ symbol) |
| To | to:dog | Posts directed at this handle (no @ symbol) |
| Hashtag | (#dogsrule) | Hashtags in parentheses, always with # symbol |
| Language | lang:en | ISO 2-letter language code |
| Replies only | filter:replies | Filter to replies only (default includes both posts and replies) |
| Min replies | min_replies:100 | Minimum reply count |
| Min likes | min_faves:100 | Minimum like count |
| Min retweets | min_retweets:100 | Minimum retweet/repost count |
| Has link | filter:links | Only posts containing links |
Twitter/X treats spaces as AND and uses negation with a minus sign rather than NOT. Examples:
| Intent | Query | Boolean equivalent |
|---|---|---|
| Either term | cat OR dog | cat OR dog |
| Both terms | cat dog | cat AND dog |
| Exclude a term | (cat OR dog) -mouse | (cat OR dog) NOT mouse |
| Combined | (cat OR dog) ferret | (cat OR dog) AND ferret |
| Complex | (cat OR dog) mouse from:parrot | (cat OR dog) AND mouse AND from:parrot |
For a full reference, see X Advanced Search.
What is Collected
Each document returned from the Twitter/X source represents a post or reply. Fields are mapped to the Datastreamer unified schema and include content text, author metadata, engagement metrics (likes, retweets, replies, quotes), and post date.
Platform-specific fields are available under the twitter schema namespace. See the Schema Reference for field details.
Troubleshooting
Job fails or returns no data
- Check that the query syntax is valid
- Verify the date range contains data
- Confirm filters are not too restrictive
- Review Job logs for specific errors
Unexpectedly high document counts
- Add engagement filters (e.g. min_faves, min_retweets) to reduce volume
- Set a document limit on the Job
- Review the DVU pricing page
Provider switch noted in Job logs
- This is expected behavior. If a provider is unavailable, the Job is routed to an alternative automatically. No action is required.
Related
- Sources Overview
- Creating Jobs
- Data Volume Units
- Direct Integrations: use if you need a specific provider directly