Search API
An Overview of what Search API offers within Datastreamer
Our full-text search API is based on Apache Lucene and provides advanced search facilities on top of a high-quality content index. If you’re getting up and running with Datastreamer for the first time you probably want to be using our Search API. This API allows you to search for arbitrary text strings, search with complex boolean logic, use filters, and other advanced features. The results are then returned as ordinary JSON documents.
Endpoint Details
Endpoint | Type | Description | Status |
---|---|---|---|
Search API | POST | Central endpoint for Search API which enables developer to retrieve relevant results based on the provided parameters. | Live |
Request Object Details
A sample request body looks like:
{
"query": {
"from": 0,
"size": 1,
"query": "New York",
"data_sources": [
"wsl_twitter"
]
}
}
Data Source , Query and API Pagination are extensively used within the JSON request body to filter out the results as per the requirements. Apache Lucene based syntax is used to write down the queries within the query
parameter of the request body.
Response Details
Based on the data_source
provided in the request body, the response of the Search API may vary as each data source have its own set of values alongside the standard datasets like Source, Content, Author, and Enrichment which are retrieved for each query result data. Here is a sample response against the above request body.
{
"results": [
{
"id": "1646766821028700399-artemis",
"data_source": "wsl_twitter",
"source": {
"link": "https://twitter.com/npnewamericans/status/1501274294852096000"
},
"content": {
"body": "#ICYMI: We celebrated #InternationalWomensDay with @TXOrgProject and the City of San Antonio with an all-women Mariachi band Las Alteñas and remarks from powerful women activists, naturalized citizens and members of the San Antonio City Council! If you missed it, watch below!",
"found": "2022-03-08T19:13:41Z",
"published": "2022-03-08T19:10:58Z",
"hashtags": [
"InternationalWomensDay",
"ICYMI"
],
"favorites": 3802,
"followers": 4043,
"following": 1929,
"mentions": [
"TXOrgProject"
]
},
"author": {
"name": "NPNA",
"bio": "<p>We represent the nation's largest immigrant coalitions. Aim to achieve vibrant, just democracy for all. Immigrants are the soul of our org, inspire our work</p>",
"location": "",
"profile_image_source": "https://pbs.twimg.com/profile_images/1501225151668445184/IUKHGkge_normal.jpg",
"gender": "UNKNOWN",
"url": "https://twitter.com/npnewamericans",
"bio_links": [
"http://partnershipfornewamericans.org"
],
"handle": "npnewamericans"
},
"enrichment": {
"sentiment": "POSITIVE",
"language": "en",
"reported_violence": {
"label": "No violence",
"confidence": 0.9921875
},
"location_inference": {
"label": "Las Vegas",
"confidence": 0.833333334
}
},
"twitter": {
"tweet_type": "POST",
"retweet_type": "REPLY",
"post_identifier": "1501274294852096000",
"user_verified": true,
"user_id": "469708047"
}
}
],
"total": {
"value": 10000,
"relation": "gte"
}
}
Updated 6 months ago