This new major update solves the issue of selecting and integrating the right provider for outcome-focused companies hosting their pipelines on Datastreamer.

"Auto" source selection is available for many major data categories, and handles the selection and ingestion of web and social data! Selecting "Auto" components offloads to the platform to select the premium 3rd party data source provider best fit for the query, and with the highest reliability based on current demand.

Using the "Auto" source selection components for any data source simplifies the process of selecting, testing, maintaining, and migrating between various providers.

Organizations who are already running their web and social pipelines on Datastreamer have already seen the benefits of: Unify (data normalization), Jobs (integrated query running and management), and a centralized usage and billing system. Now, with "Auto", the flexibility and reliability of data ingestion has gone to a new level, and is something uniquely possible only for pipelines running on Datastreamer

For more information on this new capability feature: Read More

For non-integrated sources, the reporting of used metrics was not available. With this update, we are starting to add usage metrics from various vendors inside the Job metrics.

In this first release, Jobs using Data365 will return the number of DVUs in the usage endpoints.

For more information on the Jobs Count API, Click Here

The billing dashboard within Portal has been improved to show clearer breakdowns of daily data volume costs within the platform. With this improvement, you can easily see (and filter) to the specific elements leading to your total costs.

This has also been applied to historical data available in the billing platform.

Learn more about the Billing Dashboard, Cost Explorer, and other billing support tools. More Info

Logs from components are now API-accessible, allowing you to run detailed searches across all of your logs without logging into Portal.

Using the same behavior as the Seach API from Searchable Storage; logs are accessible by specifying pipeline_logs as the data source.

More details: https://docs.datastreamer.io/docs/view-component-logs#/

A number of bug fixes have been released:

• Fixed: Content-type in webhooks is now being processed correctly.
• Improved: Logging has been improved to retain usage between versions.
• Improved: Contrast of portal colors and buttons has been increased.
• Fixed: Billing Dashboards within Portal will no longer have issues in showing data from previous months when the user is in a -UTC time zone.
• Fixed: A visual issue where long changelogs within a pipeline version would overflow the screen.
• Fixed: Product sentiment (AI Classifier) batching issue.

New ingress components for TikTok have been added to support Vetric's offerings. The new components include post searching, profile feeds, and profile details.

The new Failed Items Viewer is a built-in panel inside the Datastreamer Pipeline page that lets you inspect (and retry) failed items from individual components in your pipeline. It is available only when your pipeline is running (deployed) on Datastreamer.

All failed items are stored and you can now trigger a retry of the content after any missing configurations or requirements are solved.

Read more: https://docs.datastreamer.io/docs/failed-items-viewer/

The new "Automatic" Job type will automatically adapt to the frequency of new content being posted. After each run, the Jobs triggers a new collection for when new content is likely to appear.

After an initial run of data collection, the Job Management will track the new content frequency and automatically adapt to each previous run and adapt it's next periodic running. This will provide a cost-effective and adaptive manner to ingest data for platform users.

Released for WebSightLine Augmented Instagram for initial release, this capability will soon rollout for additional sources.

Custom Functions has received a number of features and usability improvements:

• You can now resize, expand the menu to add your Custom Functions.
• Testing the Custom Function within Portal now shows detailed results and any errors detected.
• Logging on the Custom Functions component now provides detailed error logging to assist in troubleshooting.
• You can also now use the Custom Functions to log messages into the component Logging.

A number of upgrades to the user interface have been rolled out for Portal. These updates and fixes include:

1. Logging and Metrics have been rolled out to additional components.
2. Keys & Secrets page
   1. The API keys and Secrets section are now clearly separated to ensure clarity
   2. A number of styling changes have been released to better match the overall application styling.
3. Homepage: When you enter Portal, the Pipelines page shows as the default.
4. The "Pipelines" page has had a number of UI improvements:
   1. The Pipeline ID is now visible and easy to copy.
   2. Pipeline statuses are clearer.
   3. Pipeline descriptions now show on the primary list.
   4. "Last Modified" is now visible on the Pipeline table.
5. Component Picker has now received additional styling improvements to better match the key information points that customers preferred.