Apify Setup Guide

Apify is a comprehensive cloud-based platform designed for large-scale web scraping, data extraction, and browser automation.

How to connect Apify to Datastreamer

To use the Apify Actor Integrations within your Datastreamer platform, you will need an Apify account.

1. Get your API key from the Apify platform settings

Once you create your Apify account, you need you Apify API token. To get the Apify API token, navigate to Settings > API & Integrations in the Apify Console.

2. Add your Apify API key to the Datastreamer platform's Secrets

Click the Copy icon next to your API token to copy it to your clipboard. Then, return to Datastreamer's Portal screen.

Under Keys & Secrets, add your Apify API key as its' own secret.

3. Use your Apify API key in Component configuration within your pipeline

Within your Pipeline, when you add an Apify Actor Integration component into your pipeline, you can select the secret key within the component configuration.

4. Interact with Apify Actors using Jobs

Apify Actor Integration component can be configured to run a one time job, or scheduled for multiple runs.

Jobs Setup Required Parameters:

• Actor ID
• Actor Input in JSON format
• Job Type

Example - using TikTok Data Extractor Actor

Actor ID: clockworks/free-tiktok-scraper

Actor Input: follow the instructions on the Actor 'Input' tab in JSON representation, as different Actors might have different fields.

The following Job will trigger the Actor using one keyword, and limited to 10 posts per page.

{
  "searchQueries": [
    "kanata"
  ],
  "resultsPerPage": 10,
  "excludePinnedPosts": false,
  "shouldDownloadCovers": false,
  "shouldDownloadSlideshowImages": false,
  "shouldDownloadSubtitles": false,
  "shouldDownloadVideos": false
}