In 2020, besides the first batch of “post-2000s” already born (I’m not old, I don’t listen), there are also some quietly emerging changes:
For example, the concept of smart home has emerged, and smart home products have been continuously entering daily life, bringing much fun and convenience.
Among them, smart speakers are one of the most popular smart home products, with user-friendly controls and features, connecting to WIFI, online streaming, APP control, multi-room control, etc., truly amazing!
However, the development of smart speakers is still short, with varying quality and prices, leading to statements like “smart speakers = dumb speakers”. 
Do you also have the following questions:
Is a more expensive smart speaker better? Should I buy one with a screen?
Which brands offer better cost performance? Which ones have superior performance? How to choose a smart speaker with stronger AI voice interaction?
To solve the above puzzles, guided by Shenzhen Consumer Council, the Futian District Consumer Council, Bao’an District Consumer Council, and Longgang District Consumer Council, commissioned by the Shenzhen Quality Consumption Research Institute, a comparative experiment of smart speaker products was conducted, with testing carried out by the Shenzhen Institute of Metrology and Quality Inspection, aimed at providing consumers with scientific consumption guidance and promoting quality consumption.
This comparative experiment evaluated smart speaker samples from four aspects: performance, electromagnetic compatibility, reliability, and eye safety. A total of 10 samples were evaluated, with 5 having screens and 5 without, involving brands (not ranked):
Tmall Genie, Baidu, Tencent Dingdong, Suning, Huawei, Himalaya, Xiaomi, JD Jingyuzhuo, Yeelight, Philips
The sample selection for this comparative experiment was based on consumer survey results and the popularity and sales of smart speakers on major e-commerce platforms. The samples were all purchased by staff simulating consumer purchases on various e-commerce platforms.
Testing Standards and Methods
Smart speakers belong to the audio and video electronic product category. Currently, the domestic standard for such products is GB 8898-2011 “Safety Requirements for Audio, Video and Similar Electronic Equipment”. Additionally, the electromagnetic compatibility standards include GB T 13837-2012 “Radio Disturbance Characteristics – Limits and Methods of Measurement for Sound and Television Broadcast Receivers and Associated Equipment” and GB 17625.1-2012 “Electromagnetic Compatibility – Limits – Limits for Harmonic Current Emissions (Equipment Input Current ≤ 16A)”. However, there are no mandatory national standards for evaluating the performance, response speed, and capabilities of smart speakers.
Note:
1. All samples in this comparative experiment were purchased by staff as ordinary consumers through legitimate sales channels;
2. This comparative experiment evaluated the smart speaker samples based on performance (far-field voice wake-up, false wake-up, far-field voice recognition, voice interaction response speed), electrostatic discharge, drop, and blue light hazard across seven testing items; of which, blue light hazard testing only applies to 5 models with screens, while models without screens do not require testing.
3. The evaluation results are represented by “★”; the more “★” indicates a better result, and products with the same star rating are not ranked;
4. The Philips Smart Wireless Speaker (AW6005A/93) was found to have no smart voice interaction function and is not included in this evaluation;
5. The results of this comparative experiment are only responsible for the samples purchased and are for consumer reference only; they do not represent the quality status of different batches or specifications of the same brand products and do not constitute a recommendation or advertisement for related products;
6. Without written permission from the Shenzhen Futian District Consumer Council, Bao’an District Consumer Council, and Longgang District Consumer Council, no unit or individual may use the results of this comparative experiment for commercial promotion.
Want to know the specific comparative experiment results? Research Yuan will present them to you immediately~
Smart speakers, also known as AI speakers, differ from regular speakers in that they have AI voice interaction capabilities.
The Philips Smart Wireless Speaker (AW6005A/93) is advertised as a smart speaker but was found to have no smart voice interaction function and did not participate in performance testing.
This comparative experiment set distances of 3 meters and 5 meters, and in quiet, noise, and self-noise (when the speaker is playing audio) environments, issued wake-up commands 10 times each, totaling 60 times, to evaluate the wake-up performance of the smart speakers.
From the test results, overall wake-up success rates are:3 meters > 5 meters, quiet > self-noise > noise.
The wake-up rates of the 10 samples vary significantly, with a difference of up to75%!
Among the 5 models with screens, Tencent Dingdong and Tmall Genie achieved a 100% wake-up rate, demonstrating excellent performance; among the 5 models without screens, Huawei and Xiaomi achieved a wake-up rate of 98.3%, also showing excellent performance. However, theSuning model performed poorly in far-field voice wake-up, with a wake-up rate of only25%.
▲ Far-field voice wake-up test (3 meters distance) (more wake-ups are better)
▲ Far-field voice wake-up test (5 meters distance) (more wake-ups are better)
There are “mystical” rumors about smart speakers: they may suddenly make sounds without anyone waking them up.
In this comparative experiment, 10 samples were placed in the same shielded room, and various audio files were played continuously for 8 hours to test for “interference”.
Surprisingly, all 10 samples resisted the “temptation” and did not experience any false wake-up situations, demonstrating excellent performance!
Far-field Voice Recognition
To test the smart speakers’ understanding of voice commands, we conducted a far-field voice recognition test at a distance of 3 meters, in quiet, noise, and self-noise environments, waking the smart speakers and querying the weather function, repeating the test 10 times in each scenario, totaling 30 times.
Test results showed that the far-field voice recognition capabilities of the 10 samples varied greatly, with a difference of up to22 successful queries between the highest and lowest ranked.
Tmall Genie and Xiaomi had the highest voice recognition success rates, achieving 29 successful queries; followed by Huawei and Baidu Home, achieving 27 successful queries, demonstrating excellent performance. In contrast, Yeelight and Suning achieved only 7 successful queries.
▲ Far-field voice recognition (more successful queries are better)
Voice Interaction Response Speed
According to prior consumer survey results, 83% of consumers emphasize the response sensitivity of smart speakers.
This comparative experiment was conducted in a quiet environment, evaluating the response speed of smart speakers in querying the weather and playing songs.
Test results found that the fastest and slowest samples had a response time difference of over2 seconds.
Tmall Genie had the best overall performance, with response times of 1.79 seconds for weather queries and 2.02 seconds for music playing; whereas Himalaya had the slowest response times, with times of 3.93 seconds and 4.62 seconds respectively.
▲ Voice interaction response speed – weather query (lower values are better) unit: seconds
▲ Voice interaction response speed – music playback (lower values are better) unit: seconds
Electromagnetic Compatibility
Electrostatic Discharge Testing
In many life scenarios, static electricity can accumulate, and when it reaches a certain level, it discharges, which canaffect the normal operation of electronic products and even cause equipment failure.
Test results showed that all 10 samples met the national standards for electrostatic discharge immunity testing, nice~
Most smart speakers are lightweight and easy to move, so there is a risk of dropping or damaging them during movement.
Test results showed that all 10 samples met the national mandatory drop test requirements~
Many brands have launched smart speakers with touch screens and added various parent-child and early education video content. However, many parents worry that long-term use may affect their children’s eye health.
Test results showed that the 5 smart speaker samples with screens all reached RG0 level (highest level) in blue light testing, demonstrating excellent performance.
After getting the above comparative experiment results, do you have a preferred smart speaker? After purchasing a speaker, for your and your child’s health and safety, please pay attention to the following matters~
Smart speakers should not be placed too far away
Smart speakers should be placed in a position you frequently use, with a recommended distance of 3-5 meters; being too far away will affect the smart speaker’s ability to recognize your commands.
Do not set the volume too high
When using smart speakers to play music or videos, the volume should not be set too high, as prolonged use may affect the user’s hearing health.
Ensure ventilation and keep away from heat sources
Some smart speakers have built-in rechargeable batteries, so it is recommended to place them in a well-ventilated area indoors to avoid humidity and keep them away from heat sources.
Do not watch videos on the screen for extended periods
Although some smart speakers also have video playback capabilities, most smart speakers currently have small screen sizes and low resolutions, and prolonged use of smart speakers to watch videos may negatively affect vision.
Appendix: Summary Table of the 2020 Smart Speaker Comparative Experiment Results (click to view large image)
1. All samples in this comparative experiment were purchased by staff as ordinary consumers through legitimate sales channels. The results are for consumer reference only and do not constitute a recommendation or advertisement for related products;
2. This comparative experiment evaluated the smart speaker samples based on performance (far-field voice wake-up, false wake-up, far-field voice recognition, voice interaction response speed), electrostatic discharge, drop, and blue light hazard across seven testing items; of which, blue light hazard testing only applies to 5 models with screens, while models without screens do not require testing.
3. The evaluation results are represented by “★”; the more “★” indicates a better result, and products with the same star rating are not ranked;
4. The Philips Smart Wireless Speaker (AW6005A/93) was found to have no smart voice interaction function and is not included in this evaluation;
5. The summary table indicates “√” for compliance with the reference standard requirements of this comparative experiment and “×” for non-compliance; “N/A” indicates that the product is not applicable to that test item.
6. The RG0 level in the blue light hazard test indicates that the lamp does not cause any harm under the defined limit conditions, with the blue light hazard defined as not causing harm to the retina within 10,000 seconds;
7. Brand and model references are based on product information on the sample packaging and sales page;
8. The results of this comparative experiment are only responsible for the samples purchased and do not represent the quality status of different batches or specifications of the same brand products;
9. Without written permission from the Shenzhen Futian District Consumer Council, Bao’an District Consumer Council, and Longgang District Consumer Council, no unit or individual may use the results of this comparative experiment for commercial promotion.
(Click on the product name or reply with keywords to view)
