Over the past year, flash storage has expanded into new markets and use cases due to the availability of cheaper quad-level cell (QLC) flash storage. This has allowed flash to enter array markets and use cases that were previously only suitable for nearline storage. Additionally, the price per gigabyte (GB) of flash has decreased, bringing it closer to the cost of traditional hard disk drives (HDDs). Some experts even predict the demise of HDDs and the eventual dominance of all-flash datacenters.
Enterprise flash storage refers to systems that contain multiple flash drives in rack-mounted array form factor products. These arrays aggregate the capacity of many drives, with storage access controlled by a controller that handles input/output (I/O) from hosts to the storage and manages data allocation and maintenance tasks. Enterprise flash storage arrays can range in capacity from tens of terabytes (TB) to petabytes (PB) and support block, file, or object-based storage.
QLC flash is the latest generation of flash storage media, allowing each cell in the flash chip to store four bits of data using 16 states. This higher density enables more data to be stored in the same space compared to previous flash technologies like triple-level cell (TLC) flash. While TLC drives are still prevalent in enterprise storage arrays, QLC flash has gained traction for unstructured data workloads. As manufacturers increase the number of states per cell, storage density increases while the cost per GB decreases, although higher density may affect the endurance of flash media.
Non-volatile memory express (NVMe) is a protocol specifically designed for use with flash storage. It optimizes queues and buffers for flash drives, improving performance compared to older protocols like Serial Advanced Technology Attachment (SATA) and Serial-Attached SCSI (SAS). NVMe flash drives can achieve lower latency and higher throughput rates, making them ideal for enterprise flash storage. NVMe-over-fabric technologies allow NVMe connectivity across long-distance connections within the datacenter using various protocols like Ethernet, Infiniband, and TCP.
Hard disk drives (HDDs) have been around for decades and rely on magnetic read/write heads and spinning disks. They can also be aggregated into rack-mounted arrays and are still widely used alongside flash storage. However, flash storage offers significant performance advantages compared to HDDs. Flash drives have lower latency, faster access times, and higher input/output operations per second (IOPS) when aggregated into a storage array. Flash also offers higher throughput rates, with gigabit-per-second (Gbps) rates several times faster than HDDs. HDDs have larger capacities, with models available up to around 22TB, while flash drives often come in smaller sizes due to cost considerations.
In terms of cost, flash storage has historically been more expensive than HDDs on a per-GB basis. However, flash drive costs have been declining, narrowing the cost differential. At the end of 2023, flash storage cost an average of $0.075/GB, while HDDs cost $0.05/GB for SAS and $0.035/GB for SATA drives. Some argue that storage costs should not be judged solely at the drive level, as other factors like performance and reliability also impact overall costs.
The question of whether flash will replace HDDs entirely depends on different perspectives. Some believe that flash will eventually render HDDs obsolete, with all-flash datacenters becoming the norm. Flash storage suppliers like Pure Storage predict the demise of HDDs by 2028 due to their ability to aggregate larger capacities with proprietary flash modules. However, others argue that spinning disk still has a place in certain use cases and datasets, especially in hyperscaler datacenters where HDDs are still widely used. According to HDD supplier Toshiba, approximately 85% of all data is still stored on HDDs, and the transition to all-flash datacenters may take some time.
Both flash and HDDs can be used in cloud storage, with the choice often determined by performance and cost requirements. The three major hyperscale cloud providers, Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP), offer solid-state storage options that provide a balance between cost, capacity, and performance. Cloud storage customers may not always be aware of the underlying media used in their storage, as specifications are typically based on performance criteria. However, customers can also specify flash storage in the cloud, and these hyperscalers offer flash storage options tailored to specific workloads and environments.
The concept of the all-flash datacenter has been discussed for about a decade, advocating for the replacement of HDDs and other media with flash storage. The decreasing cost of flash storage and the advantages of rapid access have propelled the idea of running analytics on larger datasets, such as using artificial intelligence (AI) on customer datasets. Flash storage also offers the benefit of quick recovery in case of a ransomware attack. However, analysts point out that while flash, especially QLC flash, has been adopted in secondary workloads, it may not replace spinning disk in all use cases. HDDs may still be useful for certain datasets, and the flash capacity to replace HDDs on a large scale is not yet available.
