Moving to Data Streams

Data Streams is the simplest way to run sourcing on Datastreamer. It manages provider selection, scheduling, and failover for you across the Auto Sources, and it uses a single committed-usage pool across all of those sources instead of a separate commitment per source. This page explains what changes when you move, what stays the same, and how to switch.

What Data Streams Is

Data Streams is the primary, managed path on Datastreamer. You configure the data you want and Datastreamer selects the best available provider for each Job through Auto source selection, with automatic failover if a provider is unavailable. It runs on the same platform, pipelines, and unified schema you already use. To connect a specific provider with your own API credentials instead, see Direct Integrations.

What Changes When You Move

• One commit across all your Auto Sources. Instead of a separate committed-usage discount per source, your commit is a single pool of DVUs that applies across every Auto Source in your account.
• Move volume between sources freely, because the commit is pooled.
• Add sources without a new commitment. Adding another Auto Source is a configuration change, not a new commit.

What Stays the Same

• The same DVU model: 1 DVU per Job Run plus 1 DVU per 100 documents, with the first 100 documents per Job Run included.
• Your pipelines, data, and unified schema.
• Enrichments, still billed separately at per-component DVU rates.
• Cost control: commits stay monthly and adjustable, and budget alerts, per-job DVU visibility, and tag-level billing all still apply.

Which Sources Are Covered

Data Streams covers the Auto Sources, which include Instagram, Reddit, X/Twitter, Facebook, LinkedIn, Threads, TikTok, YouTube, Bluesky, and Auto Profiles, with more added over time. See the Data Stream Sources Capability Table for per-source detail.

When to Use Direct Integrations Instead

Direct Integrations is the advanced path for connecting a specific provider with your own API credentials, or for provider-specific features Auto source selection does not expose. Commits on Direct Integrations are per data source. You can run both paths in the same account.

How to Move

Talk to your account manager to move your sourcing onto a Data Streams commit. Commit-level changes take effect at the start of a new billing cycle per your usage agreement, so your account manager aligns the switch to your next cycle. Your pipelines keep running throughout.

Related

• Committed Usage Discounts (Commits)
• Data Streams Overview
• Data Stream Sources Capability Table