For AI agents: visit https://docs.datastreamer.io/llms.txt for an index of all pages formatted in Markdown and endpoints in OpenAPI.

Jump to Content

View Interactive Demo ↗Book a call ↗Go to Datastreamer.io ↗

Documentation Recipes & Templates Product Updates

View Interactive Demo ↗Book a call ↗Go to Datastreamer.io ↗

Documentation

Documentation Recipes & Templates Product Updates

Getting Started

👋 Welcome to Datastreamer!
🚀 Platform Overview
🎯 Platform FAQs
What are Data Volume Units?
Platform Glossary

Creating Your First Pipeline

Creating Your First Pipeline
Creating Your Account
Quick Start Pipeline Templates
- Sentiment, Location, and Language Enriched Pipeline Template
- Multi-source Entity Recognition Pipeline Template

Pipeline Management

What is a Dynamic Pipeline?
Pipeline Deployment
Managing Pipelines
Tracking & Monitoring
Quick Diagnostics
Creating Jobs
- Creating Jobs by Importing a Jobs CSV File

DATA CONNECTORS

Connector Automation (Jobs)
Connector Automation (AI Tools)
- MCP Server Setup Guide
- Job Creation Agent
Automated Data Sources
Data Sources
Databases
- Datastreamer File Storage Ingress
- Datastreamer Searchable Storage Ingress
Event Streaming
- Google Pub/Sub Ingress
Direct Data Upload
Cloud Storage

TRANSFORMATIONS

Unify Transformer
- Unified Data Dictionary
JSON Schema Transformer
Custom Transformations

OPERATIONS & ENRICHMENTS

Overview
Custom Functions
Routing & Filtering
AI Operations
NLP Classifiers
- Language Detection (Datastreamer)
- Language Detection (Google Service)
File Operations
Content Enrichment
- X/Twitter Post Metrics Collector
- X/Twitter Retweets Collector

DESTINATION CONNECTORS

What is "Pipeline Egress"?
Databases
- Snowflake
  - Snowflake Setup Guide
- Databricks
  - Databricks File Egress
  - Databricks SQL Egress
Searchable Storage by Datastreamer
ETL Platforms
- Fivetran
  - Fivetran Setup Guide
Webhook
Firehose
- Firehose
Cloud Storage
Event Streaming
- Google Pub/Sub Egress Connector
Datastreamer File Storage Egress
Elasticsearch Egress
Big Query
- Big Query Writer
- Big Query Fixed Schema Writer
Social Voice

Billing & Cost Management

How Datastreamer is priced
Committed Usage Discounts (Commits)
Billing Management
Tips: Optimizing Your Datastreamer Usage
Add-On Bundles

Security & Compliance

Datastreamer Architecture Overview
Pipeline Regional Deployment
Platform Transparency Overview
Platform Security Overview
- API Key Regeneration
Compliance-Sensitive Usage

Other

Tips & Tricks
Google Sheet Integration
📘Executive "Launch" Guide

MCP Server

MCP

Powered by

Routing & Filtering

Utilize in-pipeline routing and filtering to automate the movement of data and application within the platform.