Metadata Fields

The primary source of information regarding metadata within API responses.

Datastreamer's pipeline contains a high-powered transformation service that can convert billions of individual fields into a common schema per minute. This unlocks the ability to easily generate features and machine learning capabilities that use multiple sources concurrently; as well easy integration of new data partners and pipeline capabilities.

View "Integrated Data Sources" and "Integrated Enrichments" to see which metadata categories apply to individual sources.

Datasource Response Bodies

Each data source has its own specific response parameters based on the source from which information is being fetched. Each data source response is merged with common information like Source, and Enrichment details.

Source

FieldDescription
source.linkCommon for all sources, this is the URL link directly to the content and can serve as a UID for the content.
source.kindFor sources with subtypes (such as dark web content), this covers the subtypes.

Content

FieldDescription
content.bodyFull-text content
content.foundWhen the content was discovered
content.publishedWhen the content was originally published
content.last_updatedWhen the content was last updated
content.hashtagsAny hashtags within the content
content.linksAny links found within the content
content.favoritesA count of social media favorites of the content at the time of index
content.followersA count of social media followers of the content at the time of index
content.followingA count of the social media following count of the content at the time of index
content.mentionsProfiles that are being mentioned in content or directed to with said pieces of content
content.image_urlsContains all URLs to images that were present within or associated to the content.
content.titleThe header (title) of the content.
content.summaryA summarization or excerpt of the content.
content.domainThe associated domain of the content record.
content.cashtagsAny cashtags within the content
content.comments_countA count of number of comments on the social media post
content.locationLocation mentioned with social media post, usually provided by the author.
content.video_urlsVideo url links within social media post [*]
content.shares_countA count of number of times social media post was shared
content.plays_countA count of number of times social media post was played
content.views_countA count of number of times social media post was viewed
content.likes_countA count of number of times social media post was liked
content.imagesContains images information associated with the content

images

FieldDescription
urlURL to image found in the content
alternative_textAlternative text or accessibility caption associated with image found in the content

Author

FieldDescription
author.nameName of the author
author.handleHandle of the user
author.bioA personal description of the author
author.locationLocation of the author
author.profile_image_sourceLink to the profile image of the author or profile.
author.genderGender of the author
author.urlURL to the author's profile
author.bio_linksURLs mentioned in the author profile
author.useridUser id mentioned in author profile
author.verifiedIs author profile verified ? Boolean True or False
author.videos_countA count of number of times author posted videos
author.phoneAuthor phone number in the author profile

Person

FieldDescription
person.nameFull name of the individual (if available).
person.handleProfile handle
person.imageAvatar of image associated with the person.
person.aliasesAny aliases
person.alternative_namesAlternative or maiden names of the individual
person.addressesAddress associated with the record.
person.countryCountry of person
person.regionRegion of person
person.place_of_birthPlace or location of birth.
person.educationEducation history
person.work_currentCurrent employment with associated title (if available), at time of document date.
person.work_historyEmployment history
person.volunteer_experienceVolunteer experience of person.
person.bioAvailable biography of person.
person.certificationsAny certifications received.
person.languagesAny languages spoken.
person.verifiedIf the profile has been verified as accurate by partner.
person.profile_create_dateCreation date of the profile.
person.dobDate of birth, of the person.
person.dodDate of death, of the person.
person.convictedConvictions (if available) of the person.
person.family_relationsAny family relations present in the result.
person.business_relationsAny business relations present in the result.
person.business_partnershipsAny business partnerships present in the result.
person.contributionsContributions and awards of the person.
person.lobbyingAny lobbying partnerships or relationships.
person.stakeholderAny stakeholder partnerships or relationships.
person.contributionsAny contributions by the person
person.certificationsAny certifications obtained by person
person.followersNumber of followers for the person
person.followingNumber of following for the person
person.headlineHeadline or title of the person
person.honorsHonors mentioned for the person
person.useriduser id for the person in the metadata
person.hashtagsHashtags related to the person
person.maiden_namePrior first or last name of the person
person.organizationsPerson working/affiliated with the organizations
person.patentsPatents obtained by the person
person.projectsProjects worked on by the person
person.publicationsPublications by the persons
person.bioBio summary of a person
person.test_scoresExam test scores of a person
person.skillsTechnical skills of a person
person.coursesCourses created or completed by person
person.urlPerson's website url

Offenses

FieldDescription
offenses.case_numberCase number of the offense. Ex: "095978201010"
offenses.case_typeCase type of the offense as defined by the source. Ex: "felony", "gross misdemeanor", "Offense Infraction"
offenses.dateDate the offense occurred. Ex: "2003-07-28"
offenses.classClass of the offense
offenses.subclassSubclass of the offense
offenses.descriptionOffense description from the source. Ex: "SEXUAL ASSAULT CHILD", "BATTERy', "AGGRAV STALKING"
offenses.degreeDegree of the offense from the source. Ex: "FELONY", "Misdemeanor", "Infraction"
offenses.levelFeld classifying offense level.Ex: "FELONY", "MISDEMEANOR", "UNKNOWN", "INFRACTION/VIOLATION/ORDIANCE/TRAFFIC"
offenses.sentence_dateDate the defendant was sentenced. Ex: "2004-06-03"
offenses.sentence_typeType of sentence as defined by the source. Ex: "fined", "failed", "probation"
offenses.sentence_descriptionDescription of the sentence.
offenses.incarceration_dateDate the defendant was incarcerated. Ex: "2007-06-03"
offenses.arrest_dateDate the defendant was arrested. Ex: "2003-07-28"
offenses.arresting_agencyArresting agency. ExL UNITED STATES MARSHALL'S SERVICE", "Elwood Police Department", "Madison County Sheriff"
offenses.court_case_numberCase number of the offense. Example: "MCRDCRTR03-0001124-008"
offenses.court_nameName of the court that tried the defendant. Ex: "CA Shasta Superior Court", "GA Dept of Corrections (Web)"
offenses.release_dateRelease date of the offender. Ex: "2019-03-26"
offenses.record_dateFeld to calculate the age of the record. Ex: "2003-02-19"
offenses.case_filing_dateDate the case was filed. Ex: "2003-02-19"
offenses.dispositionDisposition of the offense. Ex: "DISM OTHER", "DEFERRED ADJUDICATION TERMINATED", "Adjudicated Guilty", "NOLLE PROSEQUI", "DISMISSED CASEREFILED"
offenses.disposition_typeInternal field classifying the disposition. Ex: "GUILTY", "NOT GUILTY", "WITHHELD", "NOT DISPOSED", "NO VERDICT", "UNKNOWN"
offenses.disposition_dateDate the offense was disposed. Ex: "2013-03-28"
offenses.countyCounty associated with the offense. Ex: "Benton", "JACKSON", "PORTER"
offenses.conviction_jurisdictionJurisdiction associated with the offense City, State, or County. Ex: "Woodbury", "Black Hawk", "Hudson", "Jersey City Municipal Court"
offenses.conviction_dateDate of conviction. Ex: "2004-06-03"

Product

FieldDescription
product.titleThe title of the product in the product record
product.brandThe brand name of the product (ie: Nike)
product.descriptionThe description on the product record
product.listed_dateThe date the product was listed
product.priceThe listed price
product.currencyThe currency of the price, using ISO standard (ie: USD)
product.seller_nameThe displayed name of the product seller
product.seller_urlThe URL to the profile or site of the seller
product.locationThe location of the product
product.image_urlsLinks to images of the product from the product record

Organization

FieldDescription
organization.acquired_byList of organizaitions that was acquired by the Company through a merger, consolidation, combination, exchange of shares, acquisition
organization.affiliated_companiesList of affiliated company's which is a minority shareholder of another company
organization.hashtagsHashtags used for company
organization.company_typeFollowing company types : Public company, self-employed, government agency, non-profit
organization.bioDescription of the organization
organization.headquarterLocation of organization headquarters
organization.followersNumber of followers for the organization
organization.founded_dateFounded date for the organization
organization.useriduser id of the organization within metadata
organization.locationsLocation of the organization within metadata
organization.nameName of the organization
organization.parent_companyParent company of the organization
organization.imageProfile image of the organization
organization.specialitiesSpecialities of the organization
organization.taglineOrganization headline or tagline
organization.handleHandle of the organization
organization.urlOrganization website url
organization.employees_count_rangeA count of range of number of employees of the organization

Enrichment

FieldDescription
enrichment.sentimentThe extracted sentiment of the source content
enrichment.languagePer ISO: https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes#Partial_ISO_639_table
enrichment.reported_violence.labelDisplay label representing the violence: 'No violence', 'Threat of physical violence', 'Reported violence', 'Sexual violence'
enrichment.reported_violence.confidenceDisplays a score of reported violence (both directed and indirected) arising from our in-house violence classifier. Can be filtered by confidence (.confidence) of 0.0 to 1.0
enrichment.location_inference.labelDisplays the infered city location of the top 31 cities in the United states. Currently only available on English wsl_Twitter
enrichment.location_inference.confidenceDisplays a score of location inference confidence. Can be filtered by confidence (.confidence) of 0.0 to 1.0
enrichment.location_entitiesDisplays NLP-extracted locations from within the content.body of the media.
enrichment.location_entities.confidenceDisplays a score of Location entity recognition confidence. Can be filtered by confidence (.confidence) of 0.0 to 1.0
enrichment.person_entitiesDisplays NLP-extracted persons from within the content.body of the media.
enrichment.person_entities.confidenceDisplays a score of Person entity recognition confidence. Can be filtered by confidence (.confidence) of 0.0 to 1.0
enrichment.organization_entitiesDisplays NLP-extracted organizations from within the content.body of the media.
enrichment.organization_entities.confidenceDisplays a score of Organization entity recognition confidence. Can be filtered by confidence (.confidence) of 0.0 to 1.0
enrichment.hard_news.labelDisplays label of whether content has been determined as 'Hard News'.
enrichment.hard_news.confidenceDisplays a score of Hard News classification confidence. Can be filtered by confidence (.confidence) of 0.0 to 1.0

For more information on Enrichments, view the dedicated page linked in the sidebar.

Data source-specific fields

🚧

Data source-specific fields

The below sections contain fields that are not common to every data source, and apply only to specific data sources.

Twitter-specific fields

FieldDescription
twitter.tweet_typeThe type of content (POST, REPLY or RETWEET)
twitter.retweet_typeRetweet type (NONE, RAW, or QUOTE)
twittter.retweet_linkLink of the original tweet post
twitter.post_identifierThe identification string of that piece of content
twitter.user_verifiedIf the user account has been verified
twitter.user_idUnique Twitter ID of the user on Twitter (as usernames can be changed)
twitter.quote_countA count of number of times Twitter post was tweeted with quote
twitter.reply_countA count of number of times Twitter post was give reply
twitter.retweet_countA count of number of times Twitter post was retweeted
twitter.reply_linkLink of the replied post or comment

Instagram-specific fields

FieldDescription
instagram.content_typeThe type of content (comment, post)
instagram.user_idUnique Meta ID of the user on Instagram (as usernames can be changed)

Threads-specific fields

FieldDescription
threads.content_typeThe type of content (TEXT, IMAGE, CAROUSEL, or VIDEO)
threads.post_typeThe type of post (POST, QUOTE, REPOST, or REPLY)
threads.source_linkThe original post URL if post_type is REPOST, QUOTE or REPLY
threads.post_identifierThe identification string of that piece of content
threads.user_verifiedIf the user account has been verified
threads.user_idUnique Meta ID of the user on Threads (as usernames can be changed)
threads.reply_countNumber of replies at the moment this post was found.

Tiktok-specific fields

FieldDescription
tiktok.is_adTrue/False if the post is the ad
tiktok.is_official_itemTrue/False if the post is official
tiktok.is_original_itemTrue/False if the post is original
tiktok.is_share_enabledTrue/False if the post is shared enable
tiktok.is_stitch_enabledTrue/False if the post is stitch option enable
tiktok.is_activity_itemTrue/False if the post is type of activity
tiktok.is_duet_enabledTrue/False if the post is duet option enable.
tiktok.is_for_friendTrue/False if the post is for friend
tiktok.stickersList of stickers in Tiktok social media post
tiktok.duetList of duet video tiktok social media post
tiktok.musicList of music details showing title, album name, url, duration etc.

Facebook-specific fields

FieldDescription
facebook.post_typeType of Facebook post
facebook.reactions_angry_countA count of number of angry emoji reactions on Facebook post
facebook.reactions_haha_countA count of number of haha emoji reactions on Facebook post
facebook.reactions_like_countA count of number of likes emoji reactions on Facebook post
facebook.reactions_love_countA count of number of love emoji reactions on Facebook post
facebook.reactions_sad_countA count of number of sad emoji reactions on Facebook post
facebook.reactions_support_countA count of number of support emoji reactions on Facebook post
facebook.reactions_wow_countA count of number of wow emoji reaction on Facebook post
facebook.reactions_total_countA count of number of reactions on Facebook post
facebook.members_countA count of number of members on Facebook page

LinkedIn-specific fields

FieldDescription
linkedin.industriesList of industries mentioned on LinkedIn company and member profiles. This allow member/company to choose an industry that best matches their interest in company or the type of work that they do
linkedin.interests_companiesThe Interests section displays Companies, Schools, Newsletters, and Groups that a LinkedIn member is following, subscribed to or joined, as applicable
linkedin.interests_top_voicesThe Interests top voices section displays Top Voices followed, subscribed to or joined, as applicable by LinkedIn member
linkedin.is_creatorBoolean True/False field for creator mode profile setting that can help creator member to grow your reach and influence on LinkedIn.
linkedin.is_influencerBoolean True/False field showing if a member is a LinkedIn influencer who is a thought leader within a particular industry who shares organic content to a large LinkedIn audience
linkedin.is_premiumBoolean True/False field showing a company or member is premium subscriber
linkedin.interest_countA count of number of interest subscribed by member
linkedin.praise_countA count of number of times kudos on the post
linkedin.emapthy_countA count of number of times empathy or support emoji on the post
linkedin.carouselList of multiple images and/or videos in a single post
linkedin.employees_on_linkedin_countA count of number of employees of organization on LinkedIn
linkedin.open_toBoolean True/False field shows open to new job opportunities on member profile
linkedin.connections_countA count of number of member connections

News-specific fields

FieldDescription
news.global_rankRank of the news site in the world
news.country_rankRank of the news site in the country
news.word_countLength of the article, as determined by wordcount

Darknet-specific fields

FieldDescription
enrichment.hackishnessA DarkOwl-unique field, measuring the Hackishness of any piece of content.
enrichment.ccn_entities.labelUsed to search for a desired credit card number in the body of a result. Input is 6-20 digits and supports trailing wildcards.
enrichment.ssn_entities.labelUsed to search for an individual SSN record.
enrichment.email_entities.labelUsed to search for an individual email address. Note: search the keyword of "thedomain.com" to find email addresses within an entire domain.
enrichment.cryptocurrency_entities.labelUsed to search for a cryptocurrency address in the body of a result. Supported types: bitcoin, dash, ethereum, litecoin, monero, zcash.
enrichment.ipaddress_entities.labelUsed to search for an ip address in the body of a result.

Forum-specific fields

FieldDescription
forum.thread_titleThe title of the thread for the content
forum.thread_startTrue/False on if the post is the first post (head) of a forum thread.

Others

FieldDescription
doc_dateThe date of the original creation of content. While this parallels content.published in many cases, some sources that do not provide content.published. This date is always the date of document creation at its source.

What’s Next

Response examples are provided within each API detail page.