📘
Documentation Accuracy
We try to ensure every component documentation is as accurate as possible. However as we do not manage 3rd parties product features and documentation, some 3rd party documentation may not be accurate at the time of the last update. Thank you for your understanding.

Socialgist Boards Compliance provides the same search functionality as Socialgist Boards, with an additional, preconfigured PII-redaction pipeline. It uses the Socialgist Boards Search endpoint. See Socialgist's endpoint documentation for provider details.

New to Datastreamer? Start here.

👍
Unify Schema
This data source already uses Unify Schema.

Compliance Behavior

The component runs Socialgist Boards search results through a field-hashing step and then PrivateAI PII redaction. Redaction is preconfigured by Datastreamer; you do not need to add redaction fields, a filter condition, or a mapping schema in the Configure panel.

The configured redaction targets these fields when they are present:

Field	Processing
`TextHtml`	PII is detected and redacted in place.
`SubjectHtml`	PII is detected and redacted in place.
`ThreadTitle`	PII is detected and redacted in place.
`SignatureHtml`	PII is detected and redacted in place.

The current policy targets names, email addresses, phone numbers, URLs, and usernames. Detected values are replaced using entity-type markers. The original Text, Subject, and Signature fields are emptied by a later mapping step so they do not retain an unredacted copy of the content. Other fields are passed through as supplied by Socialgist, except for the configured field-hashing step.

The internal redaction condition only determines which incoming records have one of the supported text fields. It is not the search query. Use the search query and filters below to choose which records Socialgist returns. Automated PII detection may not identify every sensitive value, so assess the output against your organization's requirements.

API Key

You need a Socialgist API key to use this component. If you do not have one, contact Datastreamer Support. Select the saved Socialgist secret when configuring the component.

How to use

Socialgist Boards Compliance uses the Jobs System. Create a job on the component to define the search query and run it once or on a schedule. The search fields and job options are the same as Socialgist Boards.

Search Queries

Filters

Filter	Description
`query`	Search text. Supports Socialgist's extended query syntax and the fields `subject`, `body`, `signature`, and `ext_key`. The query is limited to 8,000 characters.
`country`	Restrict results by source country. Multiple countries are supported.
`language`	Restrict results by language.
`domain`	Restrict results by domain. Multiple domains can be comma-separated; prefix a domain with `-` to exclude it.
`filter_site_key`	External site ID from the Message Board Crawler.
`filter_forum_key`	External forum ID from the Message Board Crawler. Requires `filter_site_key`.
`max_documents`	Soft limit on the number of returned posts.

The query supports Socialgist's extended query syntax. Examples:

Phrase:

"hello world"

OR operator:

hello | world

Search specific fields:

@subject hello @body world

Example job request

This example creates a one-time search for the phrase government in English-language results:

curl --location 'https://api.platform.datastreamer.io/api/pipelines/{PIPELINE_ID}/components/{COMPONENT_ID}/jobs?ready=true' \
  --header 'apikey: <your-api-key>' \
  --header 'Content-Type: application/json' \
  --data '{
    "job_name": "{JOB_NAME}",
    "data_source": "socialgist_boards",
    "query": {
      "query": "government",
      "country": "US",
      "language": "en"
    },
    "job_type": "oneTime",
    "query_from": "2024-12-01T00:00:00.000Z",
    "query_to": "2025-01-06T00:00:00.000Z"
  }'

For more information about creating jobs, see Job Management.