API Pagination

To paginate, you can use the parameters from and size.

From: From is an integer and must be greater than or equal to 0. This offsets the starting point of the collection returned. The maximum value is 9,900

Size: Size is an integer and must be greater than 0. This limits the number of results returned from the server to a specific number of results. Size defaults to 10. Values above 100 are capped at 100 without an error.

        "size": 100,
        "from": 0

Using search_after

At the end of any result using the Search API, you will receive the "search_after" section. This designates values for the last document property in relation to the query, as well as a tiebreaker.

Using "search_after" allows for deeper pagination of content beyond 10,000 records. It also ensures the true next page of sequenced results, which is useful in rapid data sources. As such, search_after is only usable with Streaming Integrated sources.

📘

Usage Limitations

Search_after functionality is only available for stream-integrated sources. For other use cases, utilize the "from" and "size" options.

To use "search_after", pass the "search_after" values from the end of the previous response in your next request, alongside the same query. These are sort values, not a document ID.

Example result:

    },
    "search_after": [
        "1673997527000"
    ]
}

Example addition to retrieve next 100 results:

{
    "query": {
        "data_sources": [
            "twitter_auto"
        ],
        "query": "cats",
        "size": 100,
        "search_after": [
            "1673997527000"
        ]
    }
}

Keeping Paging Consistent

For consistent paging, add the keep_alive_seconds query string parameter (30 to 300) to your request. It caches a pointer to the query results for that many seconds, which prevents new data from causing duplicates or gaps between pages, and improves performance while paging.