Troubleshooting & FAQ

What is Apify, and how does it work with Datastreamer?

Apify is a cloud platform for web scraping using prebuilt “actors.” You can use Apify actors to extract data from websites and send that data into Datastreamer pipelines for transformation and delivery.

How do I connect an Apify actor to a Datastreamer pipeline?

You can view the Apify Actor Integration setup guide here: Link

Can I use any actor from the Apify Store?

Yes, most public actors can be used as ingresses. Custom actors are also supported.

Does Datastreamer support streaming real-time output from Apify?

Apify runs actors in batch mode, and does not currently offer streaming data actors. However, you can simulate near-real-time by scheduling periodic Jobs on the Apify Actor Integration. Running these Jobs at short intervals can provide similar benefits.

Can I enrich or normalize data from Apify actors after ingestion?

Yes. Once Apify data enters the Datastreamer pipeline, you can apply any enrichment, transformation, or routing rules as you would with other sources.