The complex problems we face often do not stem from the technical level, but from the rules themselves and the users’ acceptance of these rules.
Li Jun, Special Contributor to Caijing
No one expected that Amazon would be embroiled in a murder case and become the focus of the investigation.
On a day in November 2015, 47-year-old former Kansas police officer Victor Collins died in the hot tub at a home gathering organized by his friend James Andrew Bates. Whether it was an accident or murder became the first answer the police sought. How to know what happened in the house when Victor Collins died? The police’s attention turned to an Amazon Echo in Bates’s home.
Amazon Echo—Amazon’s new generation smart home control center, connects to Amazon’s artificial intelligence platform in the background and supports voice control through human-machine dialogue. By simply calling “Alexa,” users can wake up the Amazon Echo and issue control commands, such as turning on the air conditioning, calling a taxi, checking movie showtimes, or even searching for and broadcasting news based on keywords.
In other words, the Amazon Echo is in listening mode 24/7, ready to capture the sound of the owner calling “Alexa” and execute the subsequent commands. Therefore, the Amazon Echo in Bates’s home should have recorded all sounds in the house at the time of Victor Collins’s death. All these sounds were uploaded to Amazon’s cloud computing platform.
The Kansas police immediately issued a search warrant to Amazon, requesting sound data uploaded from the Echo in Bates’s home.
The case saw a tug-of-war between Amazon and the police over whether data should be provided. The outcome was that the suspect James Bates voluntarily authorized Amazon to provide the sound data to the police. The police ultimately dropped the murder charges against James Bates.
This case made the public realize that Amazon’s Echo is a bona fide household privacy information collector. Whether it’s discussions between couples about politics, economics, society, or even culinary delights, the sounds of disciplining children or intimate moments between spouses are faithfully uploaded to the cloud by Amazon’s Echo, becoming an important part of the personal profile puzzle in the era of big data. Most importantly, Amazon has never mentioned to users how long these sound data will be stored or what commercial purposes they will be used for.
(Image/Visual China)
Voice Privacy Collection Has Begun
We know that the standard operation mode of smart voice devices is to remain on and silent, and when a user calls the keyword, the smart voice device is awakened and executes the specific command that follows the keyword. According to information previously provided by Amazon, smart voice devices only save the voice information that closely follows the command keyword when they are active. When not awakened, voice information is not saved or uploaded to the cloud.
However, the Kansas case illustrates that Amazon does not completely handle customer voice information according to its statements. During the incident, Amazon’s smart voice device was never awakened, yet Amazon’s cloud still retained some voice information from that night in James Bates’s home.
Why did Amazon retain this voice information when not in an awakened state? Although Amazon has not provided any explanation on this matter, we can speculate about the future digital puzzle through some clues.
In a patent application by Amazon in 2014 for voice interaction analysis and personalized recommendations (Patent No. US14/447487), there is content that states: “I” mentioned Santa Barbara, wine, Orange County, beach, and the San Diego Zoo while talking with Laura. Amazon would then recommend maps and travel guides for Santa Barbara, beach towels, and tickets to the San Diego Zoo on the subsequent personalized e-commerce platform. This method of extracting preference information through voice recognition is called the “Voice Sniffer Algorithm.” Customer preferences, dislikes, interests, and other information are extracted, tagged, and archived through voice sniffing.
In fact, beyond personal preference information, smart home devices can gather many unexpected hidden details through sound. Firstly, emotions. Each person’s voice tone varies with different emotional states; by extracting voice, speech rate, breathing, and even crying sounds, the system can determine a person’s current state. If coughs and sneezes are included, the system can further assess the customer’s health status.
Besides voice information, sounds like toilet flushing or treadmill activity can also be used to gauge a customer’s health. Sounds from televisions, gaming consoles, and other daily activities can similarly assist the system in analyzing and judging the customer’s daily habits.
At this point, do you feel that Amazon and Google’s smart voice devices have already collected rich sound information and can create precise customer profiles and analyses? In fact, this is just a part of the enormous puzzle.
In a patent obtained by Google in 2015 for “Privacy Analysis Personalized Content for Smart Homes” (Patent No. US14/447487), an application scenario is presented: a smart home camera extracts the cover of a bedside book, identifies it as the novel “The Godfather,” and then recommends that the customer watch the same-named movie airing on Channel 5 that evening.
Do you think these personalized recommendations are inconsequential? In fact, because video can gather far more information than sound, extracting privacy information through video content is easier and more comprehensive.
Firstly, family structure, gender, age, fashion taste, style, emotions, language, and favorite activities are all easily collectible. Moreover, bedside books, guitars in the bedroom, or basketballs, even the celebrity images on your T-shirts can be recognized and recorded.
Furthermore, the division of labor, interactions between family members, and even the decision-making processes for bulk purchases and travel plans are all recorded by various smart devices. Perhaps you and your spouse were just casually discussing a minor issue with your car and considering buying a new one if you receive a year-end bonus next year. The underlying smart advertising system has already marked the price for various web pages, mobile applications, and TV ad spots targeted at you and your spouse.
Some readers might argue that although Amazon and Google have applied for these privacy information collection patents, this does not prove that they have started to collect and store information from smart devices in a non-activated state. Unfortunately, some clues indicate that these companies may have already begun to act quietly.
The Privacy Issues Behind “Coincidences”
A couple in Portland, USA, were chatting one night, and part of their conversation was saved by their Amazon Echo and forwarded as a voice file to a contact in their address book. After the incident was disclosed by the media, Amazon provided the following explanation:
The Echo was awakened by a word in the conversation that sounded very similar to the wake word “Alexa,” and then it “seemed” to hear the command “send message.” The Echo asked, “Send to whom?” At that moment, a word sounded very similar to the name of a contact in the address book. Alexa then asked again, “Send to A?” and then it seemed to hear the word “yes.” Subsequently, the voice conversation was sent to A.
All of this seems too coincidental, so much so that it is hard to believe. Moreover, even if these coincidences occurred, when Amazon Echo confirmed the input commands loudly twice (once to confirm the recipient and once for the final confirmation), the customers did not hear any of it, which is also difficult to explain.
In addition to this incident, this year media reported several cases where Amazon smart speakers inexplicably laughed, sometimes even in the middle of the night, raising deep questions about the backend operation of smart speakers.
If the cameras and facial recognition in public places are necessary for public safety, leading to some personal privacy rights yielding to public interest, then bringing smart devices and their corresponding video/audio collection and recognition systems into the home means willingly handing over household privacy to third-party smart home device companies.
What privacy are we exposing in exchange for? Personalized services? Customer discounts? Special rewards? That’s all. Compared to personal privacy security, these benefits and conveniences are trivial.
In the face of smart home devices gradually entering ordinary households, can laws and government regulations effectively protect personal privacy? The answer may be negative. Because at this stage, the government lacks effective means to regulate and protect privacy data. News cited by Xinhua in February 2018 indicated that “currently, black market transactions may be the mainstream of data trading in China. According to incomplete statistics, approximately 5.53 billion personal information leaks have occurred in the country, averaging four related pieces of personal information leaked per person; this information is repeatedly traded in the black market.” Each of us receives spam messages daily, which reflects the current ineffectiveness of government regulation.
In the relatively poor environment for personal privacy protection, even for already solved cases, government penalties are generally light. In a major case of infringing on citizens’ personal information in Zhejiang Province in 2017, over 700 million pieces of citizen information were leaked, and more than 80 million pieces were sold. The maximum fine for those involved was only 400,000 yuan.
In the recently implemented first administrative regulation of China’s express delivery industry, the “Interim Regulations on Express Delivery,” the highest fine for selling, leaking, or illegally providing user information learned during express delivery is only 100,000 yuan for severe cases. Whether from the legal framework or current judicial practices, the government’s protection of personal privacy is far from sufficient. Therefore, when we open our doors to smart home device companies, we must be extremely cautious.
Of course, it is believed that large companies like Amazon and Google will adhere to their obligations to protect customer privacy according to government regulations. However, in today’s smart device market, players are no longer just big companies like Amazon and Google.
According to Strategy Analytics’ global market data for smart voice devices, in the fourth quarter of 2016, Amazon and Google held a 96.7% market share, while other manufacturers only accounted for 3.3%. By the first quarter of 2018, the combined market share of Amazon and Google was only 70%, with companies like Alibaba, Apple, and Xiaomi rising rapidly. In addition, other manufacturers accounted for nearly 14% of the market share.
Currently, Chinese customers face a variety of smart voice device options, represented by software manufacturers like Tencent, Baidu, Alibaba, JD.com, and Cheetah, hardware manufacturers like Xiaomi, home appliance manufacturers/distributors like Haier and Suning, and startups like Out of the Door and Himalaya, among others. In the fierce competition, smart voice devices even have products priced below 100 yuan. I doubt whether products at such a price can adequately meet personal privacy protection needs.
Moreover, relying on the voluntary actions of these large and small internet companies to protect personal privacy? Given the current state of China’s internet industry, that is insufficient. From Baidu’s performance in medical information services to Didi’s lax management of customer labels, it is evident that users cannot trust the current state of privacy protection in China’s internet industry.
In addition to the proactive actions of manufacturers, the threat of the internet black market to personal privacy should not be underestimated. Recently, a big player who started with security software publicly claimed that his company has the ability to hijack the remote control of any new version of Tesla cars. Yet, videos from home cameras bearing the logo of this manufacturer are widely circulated on the black market. If a technical flaw leads to hackers hijacking smart home devices in bulk and using AI systems to extract privacy information, then all privacy information of the families using these devices would be exposed.
Unfortunately, currently, when Chinese users purchase smart voice devices, their main criteria are still functionality and price, often unaware of the personal privacy risks behind them. Even the media currently provides very few warnings about the privacy risks of smart home devices.
Rational Assessment of Trust and Transparency Mechanisms
As a rational consumer, how should one choose smart home manufacturers from the perspective of privacy protection? The core lies in two points: trust and transparency.
Trust is based on a good past business record, allowing customers to feel a sufficient level of trust in the company’s attitude and ability to protect privacy data. This entirely depends on the company’s brand-building capability.
Transparency means that the company fully discloses the principles and purposes of privacy data use and protection. In simple terms, it means the company clearly states what type of customer data is collected, why it is collected, how this data is used, with whom it is shared, and how it is shared, etc. It could even further clarify how compensation would be handled in case of a privacy data breach.
Smart home device companies can refer to the privacy protection principles for IoT devices submitted by the non-profit organization EPIC (Electronic Privacy Information Center) to the U.S. Federal Trade Commission (FTC):
* Use fair information management practices
* Employ enhanced privacy protection technologies
* Require companies to respect customers’ rights not to be tracked, profiled, and monitored
* Require companies to only collect necessary data
* Ensure companies adhere to transparency principles throughout the design and operation of IoT devices
Smart home devices are a novel phenomenon in human society. While trying to formulate detailed privacy protection and regulatory rules, we will still face very complex issues.
The complex problems we face often arise not from the technical level, but from the rules themselves and users’ acceptance of these rules. Imagine a scenario where an angry husband threatens to kill his wife with a weapon; should the smart speaker/camera in the room automatically call the police?
For the husband, he certainly would not want the smart device to call the police automatically. For the wife? The answer could be yes or no, depending on her perception of the husband’s threat’s authenticity. But regardless, this is a very difficult judgment, both for the parties involved and for the backend AI system.
For now, we can rest easy because none of the smart speakers/cameras on the market, whether from Amazon, Google, or domestic brands, provide such functionality. But if one day the government, for crime regulation purposes, requires smart home devices to provide similar functionality, would you find that acceptable?
Perhaps in that spring of 2016, at the scene of a death, whether humans are willing or not, they have already fallen into the digital age’s net, with no escape.
(Editor: Xie Lirong)
(This article was first published on June 25, 2018, in Caijing Magazine)
Everyone is watching:
Is China a competitor or a strategic enemy of the U.S.? | Interview with Wei Shangjin
Reconsidering trade disputes: How can China achieve true innovation? What is a reasonable industrial policy?
Harvard Professor Ke Weilin: Where is HNA going?
The Dilemma of Quanjude: How can a 154-year-old duck regain the hearts of young people?
After eight years, Zhang Hongli will not renew his contract as Vice President of ICBC.
Editor | Su Yue [email protected]
This article is an original piece by Caijing Magazine and cannot be reproduced or mirrored without authorization. For reprints, please leave a message at the end of the article to request authorization.
