Sentiment, Location, and Language Enriched Pipeline Template

Import this pipeline template to enrich a single data source with sentiment, location, and language.

When to use this Template?

This pipeline template is a perfect starting template for analysis of social content from a single source. With integrated language detection, filtering, Unify, Sentiment, and Location Inference; this pipeline can unlock sentiment analysis by country.

You can easily customize this pipeline to your needs.

Pipeline Workflow Design

Within this template Unify converts the data source to the Datastreamer Default schema, allowing all the following enrichments to run on default configuration. Document Deduplication then removes duplicate content coming from different queries to reduce downstream costs. Filtering removes the duplicate content, and passes for Language Detectionwhich adds language metadata fields. Further filtering from the JSON Document Router separates by language, passing the English content for Sentiment enrichment, and Location Inference which adds a country-level inference to the post.

Document Inspector is used in various locations to view the content as it passes through the pipelines as there, as no static egress (like a data warehouse or web hook) has been specified.

Pipeline Architecture Components

This pipeline uses the following components:

Ingress (1): WebSightLine Augmented Instagram
Transformations (1): Unify Transformer
Pipeline Management (3): Deduplication, Document Inspector, JSON Document Router
Enrichments (3): Language Detection, Location Inference, Sentiment

Start working with this Pipeline

To import this pipeline, save as a ".pipeline" file and import from your Pipelines screen within Portal.

🔄
Update Pipeline Access Tokens
The data source access tokens in the imported pipelines will not automatically adapt to the keys saved in your "Keys and Secrets". You will need to change them to match.

{
  "version": 9,
  "exported_date": "2025-08-07T19:28:45Z",
  "pipeline_name": "Instagram with Unify Sentiment and Location",
  "pipeline_description": "Template pipeline.",
  "steps": [
    {
      "type": "ingress",
      "component_name": "wsl-augmented-instagram-ingress",
      "step_id": "mm0r972s",
      "properties": {},
      "next_steps": [
        "b00iyafw"
      ]
    },
    {
      "type": "connector",
      "component_name": "unify-transform",
      "step_id": "b00iyafw",
      "properties": {},
      "next_steps": [
        "fz1wi4iu"
      ]
    },
    {
      "type": "connector",
      "component_name": "dedup",
      "step_id": "fz1wi4iu",
      "properties": {
        "dedup_fields": [
          "id"
        ]
      },
      "next_steps": [
        "mfr2uaum",
        "gchp2pjd"
      ]
    },
    {
      "type": "connector",
      "component_name": "language-detection-datastreamer",
      "step_id": "mfr2uaum",
      "step_description": "Detecting language, adding to separate field in metdata",
      "properties": {
        "destination_path": "enrichment.language",
        "text": "content.body"
      },
      "next_steps": [
        "px7s34g8"
      ]
    },
    {
      "type": "connector",
      "component_name": "json-document-router",
      "step_id": "px7s34g8",
      "step_description": "Separating documents by language",
      "properties": {
        "allow_multiroute": false,
        "routing_table": [
          {
            "route": 0,
            "filter": {
              "operator": "eq",
              "value_type": "string",
              "path": "enrichment.language",
              "value": "en"
            }
          },
          {
            "route": 1,
            "filter": {
              "operator": "ne",
              "value_type": "string",
              "path": "enrichment.language",
              "value": "en"
            }
          }
        ]
      },
      "next_steps": [
        "8ecrgc0j",
        "8shd32kn"
      ]
    },
    {
      "type": "connector",
      "component_name": "twitter-sentiment",
      "step_id": "8ecrgc0j",
      "properties": {
        "destination_path": "enrichment",
        "main": "content.body",
        "language": "enrichment.language",
        "condition": {
          "operator": "and",
          "conditions": [
            {
              "path": "content.body",
              "default": false,
              "operator": "exists"
            },
            {
              "path": "enrichment.language",
              "value": "en",
              "default": false,
              "operator": "eq"
            }
          ]
        }
      },
      "next_steps": [
        "w7szbewc"
      ]
    },
    {
      "type": "connector",
      "component_name": "english-twitter-location",
      "step_id": "w7szbewc",
      "properties": {
        "text": "content.body",
        "author_handle": "author.handle",
        "destination_path": "enrichment",
        "condition": {
          "operator": "and",
          "conditions": [
            {
              "path": "content.body",
              "default": false,
              "operator": "exists"
            },
            {
              "path": "enrichment.language",
              "value": "en",
              "default": false,
              "operator": "eq"
            }
          ]
        }
      },
      "next_steps": [
        "fnibhh9v"
      ]
    },
    {
      "type": "egress",
      "component_name": "inspector-egress",
      "step_id": "8shd32kn",
      "step_name": "Inspecting the Non-English Posts",
      "properties": {}
    },
    {
      "type": "egress",
      "component_name": "inspector-egress",
      "step_id": "fnibhh9v",
      "step_name": "Inspecting with Location",
      "properties": {}
    },
    {
      "type": "egress",
      "component_name": "trash-can",
      "step_id": "gchp2pjd",
      "properties": {
        "log_messages": true
      }
    }
  ]
}

❓
Need help?
If you need help with setting up, customizing, or importing this template Pipeline, just let us know!

Updated about 21 hours ago