Platform Change Log
A log of all changes to the Datastreamer platform and the versions that are related to each new addition.
This represents a log of all changes to the API and the versions that are related to each new addition.
Change Notification
Datastreamer notifies its customers about all API changes through communication from their dedicated account manager team, and in our Datastreamer Community Slack Group.
Our updates are versioned according to the taxonomy of Semantic Versioning 2.0.0, and are written as a version number of MAJOR.MINOR.PATCH.
We increment our code version based on the following:
MAJOR Version: When we make API changes incompatible with previous versions.
MINOR Version: When we add functionality in a backward-compatible manner.
PATCH Version: When we make backward-compatible bug fixes or the addition of minor fields.
| Version | Date Released | Notes |
|---|---|---|
| 6.12.3 | September 12, 2023 | ✨New & Improved: - A new task engine has been released for sources that use collection tasks to populate data sources. This task engine brings a number of fixes to the Data365 adapters and will be expanded to other task-based adapters in later iterations. - The location inference classifier for data365_twitter and Twitter official APIs can now locate posts from Thailand, Turkey, Puerto Rico in addition to above 26 countries from the city location inference model. 🛠 Fixes & Updates - Updates to data365_twitter now populates the post type metadata fields - Fixes to the darkowl_search adapter solves issues where queries would time out. - Multiple fixes for the Data365 adapter task management have been released.  Additional Information on the Task Engine Update Fixes:- Duplicate content that may have been generated by overlapping tasks is reduced in frequency. - Multiple tasks that may have caused slow collection times or backlogs can now run concurrently. - Data collection, especially for larger tasks, will now run more efficiently and faster. - Fixed task syncing issues, which may have caused some tasks to not be started. - The documented process to create tasks is not changed, all updates are performed behind the scenes within the platform. |
| 6.12.2 | August 25, 2023 | ✨New & Improved: - Location Inference classifier has received a major update, adding Puerto Rico, more English coverage, and faster updates. - Many operations and enrichments have received updates to provide compatibility to both Twitter official API adapters and Data365 adapters. - WebSightLine has released wsl_threads, providing full coverage of the new Threads' network. - WebSightLine will be retiring wsl_twitter due to new changes preventing their ongoing data offering. 🛠 Fixes & Updates - Multiple fixes for the Data365 adapter have been released. |
| 6.12.1 | August 11, 2023 | ✨New & Improved: - Emoji sentiment classifier has now been released! This brand-new classifier from the Datastreamer team detects, analyzes, and applies a sentiment tag to social media content containing emojis. 🛠 Fixes & Updates - New metadata fields have been released for the upcoming adapter for wsl_threads - Location inference has been expanded to also detect Japanese location inference in the Japanese language. - The Pipeline Builder within Portal has received multiple improvements: easier application of operations, user experience improvements, and more. - Improvements to file handling have been released. 💙 Early Access & Misc - Portal dashboards will not yet be using the new pricing or your commit. This only applies to August usage and is being worked on and targeted for a later release. |
| 6.12 | August 1-3, 2023 | ✨New & Improved: - The first release of the Pipeline Builder (Beta) is now available in Portal. Use the Builder to create and experiment with various pipeline configurations and see the output. - Emoji sentiment classifier is now live and running on select sources. Use this to filter the sentiment of content based on the emojis used within the content. Also available to be used as a post-processing operation. - You can now upload PDF and JSON files directly into the Pipeline platform for faster data integration. - New pricing and packaging has been released. Thank you all for your patience and support. - You can now connect Twitter official APIs directly to Datastreamer. Reach out to your account team for early access! 🛠 Fixes & Updates - New metadata fields have been added to the Product metadata category. - Fixes to the Twingly adapters have been released to adapt to the new changes. 💙 Early Access & Misc - Portal dashboards will not yet be using the new pricing or your commit. This only applies to August usage and is being worked on and targeted for a later release. - Interested in Threads, Amazon data, or integrating Twitter (X) official APIs? We'd love to see you in the early access! Just reach out to your account manager. |
| 6.11 | July 5, 2023 | ✨New & Improved: - OpenAI prompts (ChatGPT) can now be run on any data sources through your pipelines. - Data365 is now available as a data partner. - You can now apply Operations to the Extraction API - Sample sources are now available for Data365 and WebSightLine data sources. 🛠 Fixes & Updates - Better removal of anchor text in Unify - Fixes to Image URL identification and linking content to other posts in social media threads in Unify. - Additional metadata fields available in the Datastreamer schema have been added in preparation for upcoming new sources. - Validation has been added to the Monitored Search API endpoint |
| 6.10 | June 2, 2023 | ✨New & Improved: - ESG Classifier: This new classifier is specially trained to apply ESG labels to News content, but can be used for other long-form content as well. The ESG classifier is a 4-value label. The label value is one of ‘environmental’, ‘social’, or ‘governance’ depending on the ESG topic; or ‘none’ if no ESG topic is detected in the post. 🛠 Fixes & Updates - Spanish location inference has been expanded for Twitter data sources. - A new metadata field for "cashtags" has been added. 💙 Early Access & Misc - We will be at Collision 2023 in Toronto. Meet us there! |
| 6.9 | May 31, 2023 | 6.9 brings new Data partner catalog, beta partners, and many more changes! ✨New & Improved: - Platform Catalog: An updated platform catalog is now available at: https://datastreamer.io/platform-catalog/. This new catalog provides greater information on sources. - Sample Sources: Sample sources allow you to use a sample of a premium data source for free. Allowing you to explore, test, and try out a data source free of cost. WebSightLine sources are our launch partner, however, additional partner sources are coming soon. 🛠 Fixes & Updates - Update: Schema updates provide better social media data matching. Impacting specifically Twitter retweet, reply, and quote fields. - Count API rate limit has expanded. - Our status page (status.datastreamer.io) has pre-component subscriptions added. Allowing you to only receive notifications on the items you care about. 💙 Early Access & Misc - Data365 sources are now available in early access. Reach out to an account manager if you wish to try these sources. |
| 6.8.4 | May 17, 2023 | 6.8.4 bring Qualify of Life (QOL) updates to a number of areas: 🛠 Fixes & Updates - Updates to metadata: releases of Offenses and Product categories for the inclusion of additional data partners. - Release of new v6 -> v5 firehose adapter - Fixed issue where Monitored Search may not deliver all matched content in high-volume queries. - Fixed issue where Location Inference may be missing from some data sources. - Fixed issue where the creation of new data sources would auto-assign a name with no ability to modify. |
| 6.8.3 | March 20, 2023 | 6.8.3 brings Quality of Life updates to: 🛠 Fixes & Updates - Count API: Count API now returns data faster and with clearer error messages. - Portal: You can now view previous months in Portal's billing and dashboard section. - Location Inference: The Location Inference enrichment model is now able to predict more cities for the United States. - Partners: DarkOwl groups have been added as filters - Partners: Connecting compatible data sources has received more supporting documentation. - Partners: CoHere integration is now available. |
| 6.8.2 | February 7, 2023 | 🛠 Fixes & Updates 6.8.2 brings multiple bug fixes and optimizations for partner sources. |
| 6.8.1 | January 23, 2023 | ✨New & Improved: Update 6.8.1 brings the release of Datastreamer's own Content Similarity Clustering. It allows you to cluster (group) similar content together from a query. Try it with news content today! |
| 6.8 Amur | January 9, 2023 | The 6.8. Amur release brings a wealth of new functionality into the Datastreamer platform: ✨New & Improved: - Count API: This new API endpoint allows you to view the total number of matching search results for a query. - Country Inference: Utilize this new classifier to infer the location of a content post. Specially built for social content. 🛠 Fixes & Updates - Runaway query warning: This update for monitored search provides a notice to the user if making a high-volume monitored search. |
| 6.7 | October 17, 2022 | Run a wealth of classifiers, models and operations on any data travelling through the pipeline. Enrich, expand, and combine. ✨New & Improved: - House your own operations inside the Datastreamer pipeline - Available launch operations:Detect language Google Translate Concat Map Hard news Intent Category Sentiment Private AI - PII Redaction |
| 6.6 | September 1, 2022 | ✨New & Improved: Release of Operations API |
| 6.3 | March 22, 2022 | ✨New & Improved: Launch of Datastreamer Data Partner Network with flagship partner: Opoint Media, and relevant metadata fields. Release of Named Entity Recognition (NER) and Hard News classifiers, and relevant metadata fields. Addition of doc_date. |
| 6.2.1 | February 22, 2022 | ✨New & Improved: Addition of Highlighting functionality. |
| 6.2 | January 5, 2022 | ✨New & Improved: Our Location Inference classifier and Aggregation API Endpoints have been released, and the following fields were added to the API as a result: - enrichment.location_inference.label - enrichement.location_inference.confidence |
| Up to 6.2 | 2021 | 🛠 Fixes & Updates Summarizing the changes to 6.0.X and 6.1.X in 2021 The following fields were added to the API as additional metadata information: _content.favorites _content.followers _content.following The following fields were added in preparation for the violence classifier release. enrichment.reported_violence.label enrichment.reportedviolence.confidence The following fields were modified for greater clarity. twitter.tweettype twitter.retweettype The following fields were deprecated: _ enrichment.spam_probability The following fields were added to the API as additional metadata information: instagram.contenttype content.mentions The Violence Classifier was released. An update to our metadata has introduced a new field of: author.bio_links |
Updated 12 days ago