Logs from components are now API-accessible, allowing you to run detailed searches across all of your logs without logging into Portal.

Using the same behavior as the Seach API from Searchable Storage; logs are accessible by specifying pipeline_logs as the data source.

More details: https://docs.datastreamer.io/docs/view-component-logs#/

A number of bug fixes have been released:

• Fixed: Content-type in webhooks is now being processed correctly.
• Improved: Logging has been improved to retain usage between versions.
• Improved: Contrast of portal colors and buttons has been increased.
• Fixed: Billing Dashboards within Portal will no longer have issues in showing data from previous months when the user is in a -UTC time zone.
• Fixed: A visual issue where long changelogs within a pipeline version would overflow the screen.
• Fixed: Product sentiment (AI Classifier) batching issue.

New ingress components for TikTok have been added to support Vetric's offerings. The new components include post searching, profile feeds, and profile details.

The new Failed Items Viewer is a built-in panel inside the Datastreamer Pipeline page that lets you inspect (and retry) failed items from individual components in your pipeline. It is available only when your pipeline is running (deployed) on Datastreamer.

All failed items are stored and you can now trigger a retry of the content after any missing configurations or requirements are solved.

Read more: https://docs.datastreamer.io/docs/failed-items-viewer/

The new "Automatic" Job type will automatically adapt to the frequency of new content being posted. After each run, the Jobs triggers a new collection for when new content is likely to appear.

After an initial run of data collection, the Job Management will track the new content frequency and automatically adapt to each previous run and adapt it's next periodic running. This will provide a cost-effective and adaptive manner to ingest data for platform users.

Released for WebSightLine Augmented Instagram for initial release, this capability will soon rollout for additional sources.

Custom Functions has received a number of features and usability improvements:

• You can now resize, expand the menu to add your Custom Functions.
• Testing the Custom Function within Portal now shows detailed results and any errors detected.
• Logging on the Custom Functions component now provides detailed error logging to assist in troubleshooting.
• You can also now use the Custom Functions to log messages into the component Logging.

A number of upgrades to the user interface have been rolled out for Portal. These updates and fixes include:

1. Logging and Metrics have been rolled out to additional components.
2. Keys & Secrets page
   1. The API keys and Secrets section are now clearly separated to ensure clarity
   2. A number of styling changes have been released to better match the overall application styling.
3. Homepage: When you enter Portal, the Pipelines page shows as the default.
4. The "Pipelines" page has had a number of UI improvements:
   1. The Pipeline ID is now visible and easy to copy.
   2. Pipeline statuses are clearer.
   3. Pipeline descriptions now show on the primary list.
   4. "Last Modified" is now visible on the Pipeline table.
5. Component Picker has now received additional styling improvements to better match the key information points that customers preferred.

We've released improvements and increased documentation for the SFTP egress and ingress components. Improvements in collation types have also been created.

For additional documentation, read more here: https://docs.datastreamer.io/docs/sftp-storage-egress#/

The Databricks egress options have been expanded and split into two Databricks egress options for your Datastreamer-hosted Pipelines.

• New Databricks Files Egress: https://docs.datastreamer.io/docs/databricks-file-egress#/
• New Databricks SQL Egress: https://docs.datastreamer.io/docs/databricks-sql-egress#/

The new WebSightLine profile fetcher integrates social media profile data into JSON documents, hydrating social media posts with additional profile context.

To learn more about the details of this feature, you can access the following documentation" https://docs.datastreamer.io/docs/profile-fetcher#/