Fixes have been released for:

  • Rare error messages preventing the opening of a pipeline.
  • Components later down the pipeline no longer flag earlier pipelines as having errors, when they are simply waiting.
  • Email fields in alerts no longer require saving each email individually.
  • Removed typo from the Lucene examples in component descriptions.

This new major update solves the issue of selecting and integrating the right provider for outcome-focused companies hosting their pipelines on Datastreamer.

"Auto" source selection is available for many major data categories, and handles the selection and ingestion of web and social data! Selecting "Auto" components offloads to the platform to select the premium 3rd party data source provider best fit for the query, and with the highest reliability based on current demand.

Using the "Auto" source selection components for any data source simplifies the process of selecting, testing, maintaining, and migrating between various providers.

Organizations who are already running their web and social pipelines on Datastreamer have already seen the benefits of: Unify (data normalization), Jobs (integrated query running and management), and a centralized usage and billing system. Now, with "Auto", the flexibility and reliability of data ingestion has gone to a new level, and is something uniquely possible only for pipelines running on Datastreamer

For more information on this new capability feature: Read More

For non-integrated sources, the reporting of used metrics was not available. With this update, we are starting to add usage metrics from various vendors inside the Job metrics.

In this first release, Jobs using Data365 will return the number of DVUs in the usage endpoints.

For more information on the Jobs Count API, Click Here

The billing dashboard within Portal has been improved to show clearer breakdowns of daily data volume costs within the platform. With this improvement, you can easily see (and filter) to the specific elements leading to your total costs.

This has also been applied to historical data available in the billing platform.

Learn more about the Billing Dashboard, Cost Explorer, and other billing support tools. More Info

Stacked Bar Chart

A number of bug fixes have been released:

  • Fixed: Content-type in webhooks is now being processed correctly.
  • Improved: Logging has been improved to retain usage between versions.
  • Improved: Contrast of portal colors and buttons has been increased.
  • Fixed: Billing Dashboards within Portal will no longer have issues in showing data from previous months when the user is in a -UTC time zone.
  • Fixed: A visual issue where long changelogs within a pipeline version would overflow the screen.
  • Fixed: Product sentiment (AI Classifier) batching issue.

New ingress components for TikTok have been added to support Vetric's offerings. The new components include post searching, profile feeds, and profile details.

The new Failed Items Viewer is a built-in panel inside the Datastreamer Pipeline page that lets you inspect (and retry) failed items from individual components in your pipeline. It is available only when your pipeline is running (deployed) on Datastreamer.

All failed items are stored and you can now trigger a retry of the content after any missing configurations or requirements are solved.

Read more: https://docs.datastreamer.io/docs/failed-items-viewer/

The new "Automatic" Job type will automatically adapt to the frequency of new content being posted. After each run, the Jobs triggers a new collection for when new content is likely to appear.

After an initial run of data collection, the Job Management will track the new content frequency and automatically adapt to each previous run and adapt it's next periodic running. This will provide a cost-effective and adaptive manner to ingest data for platform users.

Released for WebSightLine Augmented Instagram for initial release, this capability will soon rollout for additional sources.