Amazon has patented technology that could let Alexa analyze your voice to determine whether you are sick or depressed and sell you products based on your physical or emotional condition.
The patent, titled “Voice-based determination of physical and emotional characteristics of users,” was issued on Tuesday this week; Amazon filed the patent application in March 2017.
The patent describes a voice assistant that can detect “abnormal” physical or emotional conditions. “For example, physical conditions such as sore throats and coughs may be determined based at least in part on a voice input from the user, and emotional conditions such as an excited emotional state or a sad emotional state may be determined based at least in part on voice input from a user,” the patent says. “A cough or sniffle, or crying, may indicate that the user has a specific physical or emotional abnormality.”
It’s not clear what ads would be sent based on a user’s emotional state, but someone who is sick might be asked if they want to buy cold medicine.
“A current physical and/or emotional condition of the user may facilitate the ability to provide highly targeted audio content, such as audio advertisements or promotions, to the user,” the patent said.
If the Amazon voice assistant determines that you have a sore threat, the system would “communicate with the audio content server(s)” to select the appropriate ad. “For example, certain content, such as content related to cough drops or flu medicine, may be targeted towards users who have sore throats,” the patent says.
Alexa might then ask, “would you like to order cough drops with 1 hour delivery?” After the order is made, the voice assistant “may append a message to the audible confirmation, such as well wishes, or ’feel better!’”
System could raise privacy concerns
Companies get patents all the time for technologies that never make it to market, so there is no guarantee this capability will be implemented in future versions of Alexa.
Amazon would have to consider the privacy implications of letting its voice assistant analyze the emotional and physical states of Amazon customers. Amazon and other tech companies last month were called to a Senate Commerce Committee hearing to testify about consumer data privacy, and senators are considering whether to write a new privacy law.
Besides analyzing your physical or emotional states, Amazon’s patent says the system would take into account the user’s browsing history and purchase history:
Embodiments of the disclosure may use physical and/or emotional characteristics of a user in combination with behavioral targeting criteria (e.g., browse history, number of clicks, purchase history, etc.) and/or contextual targeting criteria (e.g., keywords, page types, placement metadata, etc.) to determine and/or select content that may be relevant for presentation to a user.
The system would use a “voice processing algorithm” to determine a user’s emotional state. The voice analysis would be able to detect “happiness, joy, anger, sorrow, sadness, fear, disgust, boredom, stress, or other emotional states.” It would make those determinations “based at least in part on an analysis of pitch, pulse, voicing, jittering, and/or harmonicity of a user’s voice, as determined from processing of the voice data.”
The system would apply tags to each physical or emotional characteristic. Those tags may be “associated with or linked to a data file of the voice input,” and “used to determine content for presentation to the user.”
The emotion-detecting system would be tailored to each user, determining the user’s “default or normal/baseline state” so that it can detect changes that indicate that “the emotional state of the user is abnormal,” the patent says.
Amazon’s analysis would presumably be more accurate when tailored to a specific user, but the patent says the technology can also determine the emotional state of “any user” regardless of whether they normally use that device.
