Direct data Ingress (UI & HTTPS)

The HTTPS Endpoint Ingress allows you send documents through the UI or via a HTTPS endpoint.

Getting Started

To begin, add a HTTPS Endpoint Ingress component to the start of your pipeline, save and deploy it.

This component has an Upload button that will open its unique menu.

From there, you have two ways of interacting with the HTTPS Endpoint, you can either upload JSON or JSON Lines files directly, or paste/type the documents in the text area below and send them.

The text area supports either a single document as can be seen in the example text, or an array of documents, for example:

[
  {
    "id": "123456",
    "content":{
    	"body": "First document"
    }
  },
  {
    "id": "789012",
    "content":{
    	"body": "Second document"
    }
  }
]

API

After inserting a valid JSON in the text area, the Code button will become available, allowing you to see a code snippet with a curl example on how to send documents programmatically via a POST request, like the following:

curl --location 'https://portal.datastreamer.io/api/pipelines/<your-pipeline-id>/steps/<your-component-id>/documents' \
      --header 'apikey: <your-api-key>' \
      --header 'Content-Type: application/json' \
      --data \
        '[
          {
            "property": "value"
          }
        ]'