Let’s first take stock of various data storage methods:
Mechanical hard drives use magnetic disks as the storage medium:
Once written, if stored reasonably still, the lifespan is about 100 years (calculated by foreign institutions, who knows how they figured that out).
However, there are other factors to consider:
1. Currently, all external hard drives are 2.5 inches, and 2.5-inch drives are almost all SMR (Shingled Magnetic Recording). In short, when you cold store some data, operations on other data may lead to incorrect reading and writing of the earlier data (if you don’t understand, you can look at the content below regarding SMR and PMR).
2. Mechanical hard drives are most afraid of vibration, collision, and drops. They are also susceptible to interference and demagnetization from strong magnets.
In summary, the comprehensive lifespan of mechanical hard drives is about 4-7 years, but even if they fail or files are accidentally deleted, special techniques can still extract files from the disks, making their file preservation security much better than that of solid-state drives.
The difference between vertical drives and shingled drives
The left side of the diagram shows the vertical technology PMR, with clear track gaps, while the right side shows the shingled technology SMR, where tracks overlap.
The vertical hard drives we refer to are mechanical hard drives using “Perpendicular Magnetic Recording” (PMR) technology,
which was first introduced by Toshiba in 2004. Its advantages arefast random read/write speeds and longevity.
On the other hand, shingled drives were developed primarily to increase hard drive capacity.
There are only two methods to increase the capacity of mechanical hard drives:
One method is to increase the capacity of a single platter,
for vertical drives, the maximum capacity of a single platter can only reach about 1.6TB.
The other method is to increase the number of platters, with dual-platter drives being more common, and some having three platters.
You can see the shape of the hard drive, for example, in the Seagate drive below:
The red frame indicates that a deeper indentation means fewer platters, but this is not important; in any case, you can only fit three disks at most inside. If you try to fit more, it won’t work, so don’t worry about the number of platters for now.
To continue increasing capacity, the “shingled magnetic recording technology” was developed.
Shingled drives have tracks that are stacked like shingles; their main performance issue is data writing performance,
especially when frequently erasing and writing data, or when writing fragmented files, which can lead to slow speeds,
and may even cause sudden drops in speed, with severe cases leading to system crashes or game freezes.
Additionally, shingled drives change the distribution of tracks. Because the operation of erasing + rewriting data in shingled drives
requires the read/write head to write data twice (because you have to write to the shingled track, you need to lift the top track),
you need to erase and then write, which undoubtedly increases the load on the hard drive,
resulting in more noise during operation and potentially affecting its lifespan.
To improve read/write performance, manufacturers often equip them with substantial caches. Generally, these are 128MB or 256MB,
so that the data from the upper track can be temporarily stored in the cache,
then after writing to the lower track, the data from the upper track is written back.
So whenever you see marketing claiming large caches, you should be cautious; basically, any disk with a cache larger than 128MB is likely a shingled drive.
If you’re familiar with NAS, you must know Synology, which has explicitly stated on its official website that
none of its models support the use of SMR technology Western Digital Red drives (2TB to 6TB capacity models),
even though these drives are marketed as “designed specifically for NAS.”
Synology explains this decision: “Due to the performance characteristics of SMR drives, this model is only suitable for environments with lower workloads.”
QNAP, another NAS manufacturer, has similar recommendations:
Western Digital Red drives using SMR technology still appear on the hard drive compatibility list,
but it is noted that:“When using SMR-based hard drives,
if the drive runs out of cache,NAS performance may be affected.
If your NAS operates in a data-intensive environment,
we recommend using data center hard drives.“
These professional server providers advise avoiding shingled drives to prevent issues for their customers,
so for drives that require frequent read/write operations, it is best to avoid these pitfalls.
Mechanical hard drives are recommended to be enterprise drives, such as: Seagate IronWolf, Western Digital UltraStar, Toshiba MG07 series. Do not consider any 2.5-inch mechanical drives or hybrid drives.
Solid-state drives use flash memory as the storage medium:
Once written, if stored reasonably still, the lifespan is also about 100 years (emmm).
1. The storage principle of flash memory is to control the gate voltage levels to represent 0 and 1. The voltage is positively correlated with the number of electrons at the gate. Long-term storage tests the insulation layer and floating gate’s ability to retain electrons; if leakage reaches a certain level, it changes from 1 to 0.
2. The aforementioned binary situation is for SLC, which only needs to control two voltage levels: above a specific threshold Vth is 1, below is 0, which requires low control precision. MLC has four levels of control: every 25% corresponds to 00 01 10 11, requiring higher precision control. TLC has eight levels: every 12.5% corresponds to 000 001 010 … QLC has sixteen levels of control: every 6.25% corresponds to 0000 0001…. In short, the more voltage control levels, the worse the data preservation stability.
3. Flash memory has a write/erase lifespan. The strongest is SLC (100,000 write cycles), while currently, TLC (1,000 to 3,000 write cycles) is common, MLC (3,000 to 10,000 write cycles) is not prominent in the market, and QLC (500 to 1,000 write cycles) is emerging.
Converted to current home data read/write volumes, SLC can last for centuries (a few hundred years), MLC can last a lifetime (less than 100 years), TLC lasts about the same as the host (around ten years), and QLC is estimated to last three to five years.
However, you must consider the exploding data volume; ten years ago, an 80GB hard drive was sufficient, generating only one or two GB of data a month; now, 500GB mechanical and 240GB solid-state drives are everywhere, with files and emojis on WeChat and QQ generating one or two GB daily. What will it be like in another ten years?
In summary, the comprehensive lifespan of solid-state drives is about 5-8 years, but once damaged, recovering data is much more difficult than with mechanical hard drives.
Here I also recommend enterprise solid-state drives:
First, the top-tier Intel Optane chips: P4800X series (there’s not much information on Optane chips, but the ability to perform full disk read/writes 30 times a day is impressive).
SLC chips: Samsung 983Z (originally Intel also had SLC, but I couldn’t find it);
MLC chips (most of the recommendations below are better eMLC or HET-MLC):
M.2 NVME protocol: Samsung SM961 = Samsung 960PRO > Toshiba XG3
M.2 NGFF protocol: Intel S3520 > Intel S3500
(Here I do not recommend M.2 interface solid-state drives, as their read/write lifespan is not as good as the other three interfaces).
SATA interface: Intel S3710 = Samsung SM865a > Intel S3610 = Samsung SM863a
U.2 interface: Intel P3700 series, Intel 750
USB drives are fundamentally still flash memory:
Here, I recommend choosing high-end custom USB drives:
Basic: Silver SanDisk IS903 controller + HET-MLC/eMLC;
Top-tier: Silicon Motion SM2246EN controller + ASMedia 1153E bridge + SLC chips.
USB drives are not recommended for long-term data storage; they are merely transport vehicles!
I do not know much about the internal structure of SD cards,
but their stability is truly hard to gauge… I highly advise against using SD cards for long-term storage!
Finally, I recommend using SSDs for daily storage, and every year using an HDD to cold store a mechanical hard drive in a moisture-proof cabinet. The annual data storage cost for one person would be: 500 (using half of an 8TB mechanical drive) + 300 (moisture-proof cabinet) + 100 (electricity costs).
