You can now connect your pipelines running on Datastreamer to Cloudflare R2 storage.

With supporting connections to both ingress and egress components.

• Cloudflare R2 Ingress: https://docs.datastreamer.io/docs/cloudflare-r2-ingress-setup-guide#/
• Cloudflare R2 Egress: https://docs.datastreamer.io/docs/cloudflare-r2-storage-egress#/

Query validation has been added to select Vetric data sources.

This query validation makes Job creation easier by highlighting any query issues where users are using fields and formats that are not accepted.

A number of video and audio enrichment models and classifiers have been released into Preview mode. These models and classifiers from Social Voice excel at short-form and long-form internet video content.

• Social Voice Transcription: Speech-to-text transcription of voice in a video or audio file. https://datastreamer.io/catalog/social-voice-transcription-model/
• Social Voice Toxicity Classifier: Determines metrics on different toxic elements which may be present in the transcription. https://datastreamer.io/catalog/social-voice-toxicity-classifier/
• Social Voice IAB Category Classifier: Analyzes the content of the video or audio transcription and categorizes it using IAB categorization. https://datastreamer.io/catalog/social-voice-iab-category-classifier/
• Social Voice Personality Model: Analyzes the content of the video or audio transcription and returns values corresponding to different Myers–Briggs Type Indicators. https://datastreamer.io/catalog/social-voice-personality-model/
• Social Voice Brand Safety Model: Analyzes different risks to brand safety according to GARM categorizations. https://datastreamer.io/catalog/social-voice-brand-safety-model-garm/
• Social Voice Political Leaning Model: Determines the political leaning of the language used in the transcription. https://datastreamer.io/catalog/social-voice-political-leaning-model/
• Social Voice Tonality Classifier: Determines if the tone of the transcription is more personal or corporate. https://datastreamer.io/catalog/social-voice-tonality-classifier/
• Social Voice Direction Focus Classifier: Analyzes text to determine the standpoint or directional emphasis of the transcription. https://datastreamer.io/catalog/social-voice-direction-focus-classifier/
• Social Voice On-Screen Text Detection Model: Detection of text on-screen in a video scene or image. https://datastreamer.io/catalog/social-voice-on-screen-text-detection-model/
• Social Voice On-Screen Logo Detection Model: Detection of brand logos in a video scene or image. https://datastreamer.io/catalog/social-voice-on-screen-logo-detection-model/

The OpenAI text prompt completion model has been updated to match the "Secrets" patterns of adding API keys in a secure manner within the platform.

A number of text enrichment models and classifiers have been released into Preview mode. These models from Tisane excel at analysis of forum, chat, and live conversation analysis.

• Tisane Sentiment Analysis: Tisane sentiment analysis supports 35+ languages, including slang and obfuscated text. https://datastreamer.io/catalog/tisane-sentiment-analysis/
• Tisane Entity Extraction: Detect mentions of people, organizations, locations, filenames, phone numbers, crypto addresses, and more. Tisane extracts both standard entities and those relevant to trust & safety/law enforcement applications. https://datastreamer.io/catalog/tisane-entity-extraction/
• Tisane Topic Extraction: Also known as: theme identification, subject detection, or key topic recognition. Topic extraction determines the dominant topics in the text. Supports IPTC, IAB, and Wikidata IDs. https://datastreamer.io/catalog/tisane-topic-extraction/

Support has been released for Hashtags and Keywords within Augmented Instagram

New capabilities for Searchable Storage have been released. Trend Aggregation uses both query defined foreground and overall background data from within your storage to detect unusual patterns. This provides an aggregated view of those unusual results with greatest deviation from the background information.

This new Trend Aggregation powers use cases of detecting trending or emerging metadata fields from across your stored content. Perfect for monitoring, listening, and trend detection use cases.

More information is available in our documentation here: Searchable Storage Technical Documentation

Fixes have been released to all Data365 connectors improving queue size validation and performance improvements.

Direct Data Upload allows you send documents through the UI or via a HTTPS endpoint. This allows you to easily integrate new data sources, without building custom configurations.

Using this component, you can add JSON (and JSON Lines) directly in Portal or through API. In additional, you can send documents programmatically via a POST request directly into your pipelines.

Fixes have been released to:

• Job retry logic has been improved to better detect failed Jobs and restart at the most optimal location.
• Pipeline validation has been expanded to warn users when trying to deploy a pipeline with missing 3rd party API keys.
• Query validation has been expanded to more social sources.