Twitter/X Setup (Auto Sources)

What are Auto Sources?

Auto Sources enables you to collect data from platforms using Datastreamer's intelligent provider selection system. Instead of choosing a specific data provider, you define your data requirements and Auto automatically selects the best provide(s) to fulfill your request.

For more details on Auto Source Selection: Read More

Troubleshooting

Job fails or returns no data

Verify your requirements are not too restrictive (may exclude all providers)
Check that the date range is valid and contains available data
Review Job logs for provider-specific errors

Unexpectedly high costs

Review the number of documents collected in your Job
Verify filters are correctly excluding unwanted data
Consider using Committed Usage Discounts for predictable, high-volume jobs

Provider switch detected in logs

This is normal behavior and indicates Auto mode's resilience at work
Your data collection continues transparently
No action is required

Twitter Advanced Search Logic

Auto sources adapt your query to the providers' query logic and language. As a result, the basis of Twitter/X's Advanced Search formatting is used.

Here is a quick reference on creating advanced search queries. For greater details, you can view X's advanced search on their platform at https://x.com/search-advanced

⚠️
Note: Dates are not to be included in the query.
As the Job system handles the scheduling, range, and selection of the right providers for historical searches, including dates inside the query may cause Job completion errors or selection to be limited to few providers.

Item	X Example	Explanation
Phrases	"dog house"	Phrases in which the exact format of the word is expected.
NOT	-mouse	Negation, not including this word
From	from:cat	Content from these handles (note the lack of @ symbol)
To	to:dog	Content to these handles (note the lack of @ symbol)
Hashtags	(#dogsrule)	Hashtags are nested in parantheses, and unlike some profile usages, always require the # symbol.
Language	lang:en	Filter content to be from a specific language using the ISO 2-letter standard. (Note, only a few languages are supported by X, and this list has been known to change.)
Reply Filtering	filter:replies	Filter to only replies (default is replies and posts)
Min Replies	min_replies:100	Filter to only showing results with a minimum of 100 replies
Min Likes	min_faves:100	Filter to only showing results with a minimum of 100 likes
Min Retweets	min_retweets:100	Filter to only showing results with a minimum of 100 retweets (reposts)
Link post toggle	filter:links	Only show posts with links (default is both with and without links)

X/Twitter uses alternate Boolean-like logic as part of the core search logic. Examples to help guide are below:

Understanding X's Format	X Query Structuring	Boolean Equivalent
Using negation	(cat OR dog) - mouse	(cat OR dog) NOT mouse
AND vs OR treatment	(cat OR dog) ferret	(cat OR dog) AND ferret
Spaces are treated as AND	cat dog	cat AND dog
More complex query	(cat OR dog) mouse from:parrot	(cat OR dog) AND mouse AND from:parrot