API Pagination

To paginate, you can use the parameters from and size.

From: From is an integer and must be greater than or equal to 0. This offsets the starting point of the collection returned. The maximum value is 9,900

Size: Size limits the number of results returned. It defaults to 10 and is clamped to the range 0 to 100. Use 0 to return aggregations without documents.

        "size": 100,
        "from": 0

Using search_after

A Search response may include search_after values for the last result. The values correspond to the sort order and may include a tiebreaker.

Using "search_after" allows for deeper pagination of content beyond 10,000 records. It also ensures the true next page of sequenced results, which is useful in rapid data sources. As such, search_after is only usable with Streaming Integrated sources.

📘

Usage Limitations

Search_after functionality is only available for stream-integrated sources. For other use cases, utilize the "from" and "size" options.

To use "search_after", pass the "search_after" values from the end of the previous response in your next request, alongside the same query. These are sort values, not a document ID. Preserve their JSON types.

Example result:

    },
    "search_after": [
        "1673997527000"
    ]
}

Example addition to retrieve next 100 results:

{
    "query": {
        "data_sources": [
            "twitter_auto"
        ],
        "query": "cats",
        "size": 100,
        "search_after": [
            "1673997527000"
        ]
    }
}

Keeping Paging Consistent

For consistent paging, add the keep_alive_seconds query string parameter (30 to 300) to your request. The response can include a continuation_token for the point-in-time snapshot. Pass that token back inside query on subsequent requests, along with search_after, to keep paging through the same snapshot.