Firehose Egress Component

Firehose Egress Component delivers pipeline data by polling Content API

Content API

Get Pipeline Content

Retrieves pipeline content, with manual content acknowledgment, which requires a subsequent request to confirm data retrieval.

curl --location 'https://api.platform.datastreamer.io/pipeline-gateway/<your-pipeline-id>/<your-step-id>/api/content?autoAck=false' \
--header 'apikey: YOUR_API_KEY' \

💡

Note that when retrieved content is not acknowledge with '60' minutes, content will become once again available for retrieval

Alternatively, it is possible to retrieve content with no subsequent confirmation request, by setting autoAck query parameter to true

curl --location 'https://api.platform.datastreamer.io/pipeline-gateway/<your-pipeline-id>/<your-step-id>/api/content?autoAck=true' \
--header 'apikey: YOUR_API_KEY' \

Acknowledge Pipeline Content

Using ack_id value returned with Get Pipeline Content request, confirm data retrieval by sending back obtained Ack ID

curl --location --request PUT 'https://api.platform.datastreamer.io/pipeline-gateway/<your-pipeline-id>/<your-step-id>/api/content/:ack_id/ack' \
--header 'apikey: YOUR_API_KEY' \

Decline Pipeline Content

In case to decline retrieved content, and making it available for retrieval again, send back obtained Ack ID

curl --location --request PUT 'https://api.platform.datastreamer.io/pipeline-gateway/<your-pipeline-id>/<your-step-id>/api/content/:ack_id/nack' \
--header 'apikey: YOUR_API_KEY' \

Component Configuration

From pipeline Components Picker, find Firehose Egress component, and add it to your pipeline.