How To Enrich Your Own Data (Bring Your Own Data)

You can build pipelines to enrich your data with just a few steps.

In this example we are going create an pipeline to enrich content using a Product Sentiment operation.

1. First, go to Datastreamer Portal, you will need to create an account if you don't have one.

2. Go to Dynamic Pipeline and click on "New Pipeline" button.

3. Add the ingress HTTPS Endpoint Ingress no config is required.

   

4. Click on the plus (+) button below the ingress box and select the Product Sentiment Classifier from the menu. In the "Target Text" field, specify the name of the field containing the text you want to analyze. For example, in this guide, the field is named "review_text."
   The result of the operation will be saved to the destination path enrichment.product_sentiment. You can customize this path to suit your needs. If the specified field does not already exist, the component will automatically create it for you.
   Finally, use the filter condition to select the content that should be enriched. The data that do not match the criteria will not be deleted.

   

5. The last component in our pipeline is the Document Inspector, which allows you to view the content directly in the user interface (UI). No additional configuration is required.
   The catalog offers other Egress options, such as Datastreamer Searchable Storage, Webhook, Google Cloud Storage, Azure Blob, S3, Elasticsearch, and Pubsub.
   Your pipeline should resemble the example shown in the image below.

   

6. Click on the Save button to save your pipeline. Next, click on Deploy to start your pipeline, this action should take a few seconds.

   📘

   Did you know? Your pipeline runs on an dedicated environment Security

7. Once your pipeline has been Deployed, click on the Upload button of the HTTPS Endpoint Ingress.
   You can copy and past the content below or save it as a JSON file and upload it. The Code button generate a curl sample to upload the content.

   JSON

   [
       {
           "review_id": "001",
           "product_id": "A12345",
           "reviewer": {
               "name": "John Doe"
           },
           "rating": 4.5,
           "review_date": "2024-12-18",
           "title": "Excellent quality and design",
           "review_text": "The product exceeded my expectations! The quality of the material is outstanding, and the design is sleek and modern. Delivery was prompt, and the customer service was responsive. Highly recommend!",
           "verified_purchase": true,
           "likes": 15
       },
       {
           "review_id": "002",
           "product_id": "B67890",
           "reviewer": {
               "name": "Jane Smith"
           },
           "rating": 2.0,
           "review_date": "2024-12-16",
           "title": "Not worth the price",
           "review_text": "The product is overpriced for what it offers. The build feels cheap, and it stopped working within a week. I contacted support, but the response was slow and unhelpful. Disappointed with this purchase.",
           "verified_purchase": true,
           "likes": 3
       }
   ]
   

8. After a few seconds, you should see your enriched content in the Document Inspector.

Here is the same content with the enriched data.

{
  "review_id": "002",
  "product_id": "B67890",
  "reviewer": {
    "name": "Jane Smith"
  },
  "rating": 2,
  "review_date": "2024-12-16",
  "title": "Not worth the price",
  "review_text": "The product is overpriced for what it offers. The build feels cheap, and it stopped working within a week. I contacted support, but the response was slow and unhelpful. Disappointed with this purchase.",
  "verified_purchase": true,
  "likes": 3,
  "enrichment": {
    "product_sentiment": {
      "brands": [],
      "confidence": 0.95,
      "entities": [
        "product",
        "support"
      ],
      "reason": "The customer expresses negative sentiment due to the product being overpriced, poorly built, malfunctioning, and unhelpful support.",
      "sentiment": "negative"
    }
  }
}

{
  "review_id": "001",
  "product_id": "A12345",
  "reviewer": {
    "name": "John Doe"
  },
  "rating": 4.5,
  "review_date": "2024-12-18",
  "title": "Excellent quality and design",
  "review_text": "The product exceeded my expectations! The quality of the material is outstanding, and the design is sleek and modern. Delivery was prompt, and the customer service was responsive. Highly recommend!",
  "verified_purchase": true,
  "likes": 15,
  "enrichment": {
    "product_sentiment": {
      "brands": [],
      "confidence": 0.95,
      "entities": [
        "product",
        "quality",
        "material",
        "design",
        "customer service",
        "delivery"
      ],
      "reason": "The customer expressed high satisfaction with the product's quality, design, delivery, and customer service.",
      "sentiment": "positive"
    }
  }
}