📘

Documentation Accuracy

We try to ensure every component documentation is as accurate as possible. However as we do not manage 3rd parties product features and documentation, some 3rd party documentation may not be accurate at the time of the last update. Thank you for your understanding.

Integrated Add-Ons

Integration Details

All WebSightLine sources are powered by the Jobs System, and select sources also have the ability to integrate using a Live Feed.

You also have the option to use the API. You can use the Code button to extract this example:

curl --location 'https://dev.api.platform.datastreamer.io/api/pipelines/{PIPELINE_ID}/components/{COMPONENT_ID}/jobs?ready=true' \
      --header 'apikey: <your-api-key>' \
      --header 'Content-Type: application/json' \
      --data \
        '{
          "job_name": "96e4322d-6758-447b-b3a6-d7a0bff54f94",
          "data_source": "wsl_instagram",
          "query": {
            "query_string": "cats OR dogs"
          },
          "job_type": "periodic",
          "schedule": "0 0 0/6 1/1 * ? *",
          "max_documents": 50
        }'

For more details on creating data collection jobs, see Job Management.

Additional details for Augmented sources

The most advanced WebSightLine solution, this component allows you to configure jobs that search directly by user or location.

Search for users

You can either choose to search by user handle (called Profile) or by ID (called User ID)

You also have the option to use the API. You can use the Code button to extract this example

curl --location 'https://dev.api.platform.datastreamer.io/api/pipelines/{PIPELINE_ID}/components/{COMPONENT_ID}/jobs?ready=true' \
      --header 'apikey: <your-api-key>' \
      --header 'Content-Type: application/json' \
      --data \
        '{
          "job_name": "3047ca1d-b38b-48c7-947c-b0d2a8a7b8eb",
          "component_name": "wsl-augmented-instagram-ingress",
          "data_source": "wsl_instagram",
          "query": {
            "type": "location",
            "terms": [
              "cristiano",
              "leomessi",
              "therock"
            ]
          },
          "job_type": "periodic",
          "label": "Popular profiles",
          "schedule": "0 0 0/12 1/1 * ? *"
        }'

For more details on creating data collection jobs, see Job Management

Search for locations

When searching for locations, it's the same process as searching for profiles, but you must provide the location ID, not the location name, for example, if looking for Bolder, Colorado (instagram.com/explore/locations/46519841/), the ID is 46519841.

WebSightLine Source Details

The below sources are available for no-code integration.

WebSightLine Augmented Instagram

The most advanced WebSightLine solution, this component allows you to configure jobs that search Instagram directly by user or location.

It augments the results by combining live data with historical content from the WebSightLine database, delivering the most comprehensive coverage available. You can configure either one time searches or recurring searches.

WebSightLine Instagram

This component triggers jobs that will search WebSightLine's database for content discovered that matches the Lucene query provided. You can configure either one time searches or recurring searches.

The WebSightLine Instagram component offers search capabilities within a repository containing millions of Instagram posts and comments from the past two years.

WebSightLine Threads

The WebSightLine Threads component provides a live feed with millions of Threads posts and comments each day.

Additional Guides

Migrating to WebSightLine's Augmented sources

Augmented Instagram migration from other providers or other offerings from WebSightLine can be performed easily.

The schema provided by the Augmented sources is the same as non-augmented sources, and the delivery within Pipelines also shared the same configurations. The only different users may experience is in the creation of their Jobs and their behavior.

As a result, the difference in Jobs lies in requesting a specific Search Type and Search Terms.

📘

Augmented sources combine multiple origins

As Augmented sources combine newly-captured data with content indexed from other mechanisms, the Job creation is simplified, and therefore will not accept detailed filtering.

Tips and Tricks

1. To get the most out of augmented sources, use "Periodic" Job logic to have your Pipeline automatically capture and track deduplication of the sources if performing regular searching.
2. Due to the performance of sources like Instagram in archiving content, it is suggested to search more frequently in situations where more results are expected.