Direct Integrations

Direct Integrations give you access to individual data providers, cloud storage systems, and other external services through dedicated connectors. Unlike Data Streams, which manage provider selection automatically, Direct Integrations require you to configure the provider directly.

When to Use Direct Integrations

Direct Integrations are appropriate when you need to:

• Use your own API credentials with a specific provider
• Access parameters or features specific to one provider that are not exposed in standard Data Stream sources
• Connect to a cloud storage system for data ingress or egress
• Meet a procurement or compliance requirement that specifies a particular vendor
• Build a pipeline around a data source that is not available as a Data Stream source

For most use cases involving social media and web content, Data Streams are the recommended starting point. Direct Integrations are intended for teams with specific provider requirements.

What is Available

Direct Integrations are organized into the following categories:

Cloud Storage Connect to cloud storage providers for both data ingress (bringing data into a pipeline) and egress (delivering data out). Each provider page covers both directions.

• Amazon S3
• Azure Blob Storage
• Google Cloud Storage
• Cloudflare R2
• SFTP

Data Providers Individual connectors for third-party data providers. These require provider accounts and, in most cases, your own API credentials.

• Brightdata
• Socialgist
• Data365
• Apify
• DarkOwl
• Opoint
• WebSightLine
• Private Data Sources

Pricing

Direct Integrations use a different pricing model from Data Streams. Usage is measured in DVUs, but rates are calculated per component using each provider's native metric (documents, records, tokens, bytes, etc.).

See Direct Integrations Pricing for the full details, including per-component DVU conversion rates.

Adding Enrichments

Enrichments (sentiment, entity recognition, etc.) can be added to pipelines built on Direct Integrations. Enrichment DVU rates are the same regardless of whether you are using Data Streams or Direct Integrations.