Get Started

WebSightLine Threads

WebSightLine (WSL) Threads is a high-sampling of near-time public Threads content.

The WebSightLine Threads component provides a live feed with millions of Threads posts and comments each day.

New to Datastreamer? Start here.

👍

Unify Schema

This data source already use Unify Schema.

How to use?

The WebSightLine Threads is powered by the Jobs System, when interacting with the component you have the option to define your jobs queries.

Search Queries

Filters

Available filters for WebSightLine Threads can be found in the table below:

Filter NameDescription
queryList of keywords or a phrase to search
max_documentsSet a limit for the number of posts that will be fetched for the search.

The Lucene Query is supported for this component in the query field. Here are some of the basics queries that you can try:

Keywords:

cats

Fields:

title:lucene

Phrases:

"apache lucene"

Wildcards:

tes\*

Boolean operators:

cats OR dogs

Examples

Search for cats or dogs

Query cats or dogs every 6 hours:

You also have the option to use the API. You can use the Code button to extract this example:

curl --location 'https://dev.api.platform.datastreamer.io/api/pipelines/{PIPELINE_ID}/components/{COMPONENT_ID}/jobs?ready=true' \
      --header 'apikey: <your-api-key>' \
      --header 'Content-Type: application/json' \
      --data \
        '{
          "job_name": "8740eace-160a-468e-a2f5-8d2db803f9f6",
          "data_source": "wsl_threads",
          "query": {
            "query_string": "cats OR dogs"
          },
          "job_type": "periodic",
          "schedule": "0 0 0/12 1/1 * ? *",
          "max_documents": 50
        }'

For more details on creating data collection jobs, see Job Management.

Additional Details

Stats

Searchable RecordsUpdate FrequencyPartner Type
45 million (3 months)Near-time (Max 10-minute latency)Stream Integrated

Compatible Metadata Fields

Applicable Metadata CategoriesCompatible
SourceYes
ContentYes
AuthorYes
PersonNo
EnrichmentYes
OrganizationNo
Data source-specific fields?Yes, please see the Metadata page.

Compatible Classifiers & Models

Classifier & ModelCompatible
Named Entity RecognitionNo
Location_InferenceYes
LanguageYes
Reported_ViolenceNo
SentimentNo
Hard_NewsNo

Compatible Features

As a Stream-Integrated partner, all streaming features are available.

FeaturesCompatible
Search APIYes
Date HistogramsYes
Term AggregationsYes
HighlightingYes
Fuzzy and Proximate SearchYe

Built-In Language Detection

WSL_Threads has by default a built-in language enrichment provided by WebSightLine, the languages currently supported are:

ISO 639-1Language
ARArabic
BGBulgarian
CSCzech
DADanish
DEGerman
ELGreek
ENEnglish
ESSpanish
ETEstonian
FAPersian
FIFinnish
FRFrench
HEHebrew
HIHindi
HRCroatian
HUHungarian
IDIndonesian
ITItalian
JAJapanese
KOKorean
MSMalay
NLDutch
NONorwegian
PLPolish
PTPortuguese
RORomanian
RURussian
SLSlovenian
SVSwedish
THThai
TRTurkish
UKUkrainian
VIVietnamese
ZHChinese
UUndefined