Digital Storage And Memory Projections For 2025, Part 2

During recent prior year projections for digital storage and memory we have not talked much about digital storage using optical recording. Optical storage media was a big consumer technology in the past as a method for digital content distribution but there was another market for optical storage for archival storage, where cartridges of optical discs were kept in libraries, similar to the optical libraries used for magnetic tape storage.

These libraries used optical discs which were derived from Blu-ray and other mass market optical storage technologies and were often write-once optical media meant for archive applications. Let’s talk about where optical storage fits into the storage hierarchy and about the technologies being introduced by a number of optical storage startups, which could change the way that data centers store archived information.

Data centers often use a combination of digital storage products with different performance versus cost trade-offs together to meet the needs of different workflows with the highest efficiency and lowest costs. The different layers in such a storage hierarchy today often feature solid state drives, SSDs, using NAND flash as a faster but more expensive, in $/TB, primary storage. This primary storage directly feeds the faster but even more expensive and volatile, DRAM memory used to directly support computation.

There is another layer of digital storage, often called the secondary storage layer where lower cost, lower performance digital storage technologies are used to store a larger amount of less active data than that stored in primary storage. Secondary storage is where nearline hard disk drives, HDDs, are used the most to store data which has value, but which does not require the performance characteristics of most enterprise SSDs.

In addition to HDDs there are a number of storage systems meant for secondary storage using quad-level cell, QLC, flash SSDs. These SSDs often higher storage capacity at lower cost than multi-level cell, MLC, and three-level cell, TLC, NAND flash, but they have lower endurance and lower performance, but still higher than that of individual HDDs. However, the raw storage costs of these QLC SSDs are still higher than that of nearline HDDs and so these products will likely have the most use where in-frequently written secondary storage with higher performance is needed.

The final layer in this storage hierarchy is archival storage, where data is kept that still has potential value but which is very seldom accessed, for instance for various types of legal compliance or for historical value. This layer generally requires even lower costs for storage than the secondary storage layer and this is the realm where magnetic tape and optical disc storage have their greatest use. The chart below, from my colleague, Jim Handy, of Objective Analysis shows a log-log plot of storage performance versus capacity price for various memory and digital storage technologies.

Coughlin Associates projects that annual storage capacity shipments for solid state drives (SSDs), hard disk drives (HDDs) and magnetic tape/optical storage should grow over 5X from 2024 through 2029. The majority of this stored data is for secondary or active archive applications where data is retained for later access and it is generally transferred to faster SSDs and DRAM for primary storage and immediate use. The percentage of data in secondary and archived storage is growing faster than primary storage.

Optical storage libraries are still being sold as an alternative to magnetic tape storage and with the technologies being introduced by numerous startup optical storage startups, this use could increase. Coughlin Associates recently wrote a white paper on current archive storage technologies, including these newer optical storage technologies. This article includes some material from that white paper as well as the IEEE Mass Storage Roadmap.

As secondary and archive storage demand grows, there may be opportunities for new storage media to play a role, particularly if they can provide higher performance, lower cost, including ongoing energy costs, and greater longevity than current archive storage media.

There are also archival optical approaches that don’t use optical discs but rather ceramic coated rectangular glass sheets. Modern optical recording technologies are particularly interesting for archive and particularly digital preservation applications because the various optical storage media can last 100 years or more in a much less controlled environment that magnetic storage technologies.

Conventional optical media uses lasers with a fixed wavelength to ablate regions of the media or to change the reflective characteristics of the media to create optical features that can be encoded to store digital information. This is how CDs, DVDs and Blu-ray discs work.

Over 10 years ago, Panasonic and Sony created a roadmap for multiple layer two-sided Blu-ray derived optical storage that they projected would reach at least one TB per disc (Archival Disc, AD). In practice this technology, while used in some facilities, never became mainstream for general data archiving and demonstrated product disc storage capacity have not exceeded 500GB (their second-generation product, introduced in 2020. Sony and Panasonic both introduced optical disc library systems using their AD technology.

There are many other optical recording media, which include PiqlFilm, which uses black and white, negative silver-halide film on a polyester base. This method is unique in that it can include recorded human readable instructions and file format and source code for reading data, in addition to digital data. This archive method is being used in some scientific, engineering and historical archives. PiqlFilm is also the initiator of the Arctic World Archive (AWA), a repository for world memory located in Northern Norway on the Arctic Ocean.

Microsoft and the University of Southampton have explored volumetric archive recording in fused silica using a fast laser. This is a write once method with a media that should be stable for 100’s of years and can store up to 360TB of data on a 5-inch glass plate. The Southampton researchers have recently started a company, SPhontonix, to commercialize this technology. A California-based startup, Group 47 developed the Digital Optical Tape System (DOTS) that it says can preserve data for more than 200 years using a phase change media sputtered on a polyester-based film. The image below shows the SPhotonix optical storage concept.

Various holographic optical recording technologies have also been developed over the years, but, although research on this technology continues, no commercial products using holographic recording have been successfully implemented for archival storage.

Folio Photonics is a company which has developed a multilayer optical recording system for archiving applications which can result in photothermal recording similar to Blu-ray optical discs with either reflective or fluorescent recording technologies. Their technology uses photosensitive dyes dispersed in a polymer matrix to create a reflective or fluorescent optical media. This photosensitive material has a strong optical absorption at 405nm, the wavelength used for conventional Blu-ray discs.

Using roll-to-roll co-extrusion processes, multiple layers can be produced all at once in a media which can be cut and placed on plastic optical disc substrates, reducing the costs of making a multiple layer optical media. Rather than using a spiral tracking feature embossed on conventional optical discs (hence the land and grooves), Folio does head tracking with a separate laser focused on the disc substrate where the tracking pattern is embossed.

Folio Photonics believes that this could be done in multiple layers (dozens of layers all at once) with an initial manufacturing cost of less than $5/TB and with more layers going to less than $2/TB by 2029. Note that one of the on-going issues with multi-layer optical media has been the optical absorption of the layers which can reduce the storage capacity and signal from lower layers. The image below shows Folio Photonic’s projection for optical disc storage capacity with their technology.

Another interesting optical recording contender is Cerabyte, founded in 2020 and with headquarters recently moved to Silicon Valley. Cerabyte uses sputtered 10nm ceramic layers on a glass substrate. Data is written encoded in an array of data matrices using a 2-D digital micro mirror with up to 2 million elements simultaneously written by femtosecond laser pulses in the UV spectrum with a write speed of 1GB/s and with less than 1W average power. Reading is done at GB/s data rates using high-speed image sensors and parallel high speed image processing for decoding. The image below shows Cerabytes digital recording concept.

Both reading and writing are done across the square substrate by scanning the microscope optics using high-speed XY stages kept in focus using a piezo driven auto focus system enabling random data access. The 9X9 cm media sheets can be recorded on both sides and stacked in cartridges for robotic access similar to that used for conventional optical and magnetic tape libraries. Cerabyte projects media costs below $1/TB by 2030.

Optera Data’s storage technology takes advantage of changes in the optical absorption/emission characteristics of its recording media at several adjacent laser frequencies during writing and then by reading these changes. These changes are referred to as spectral holes.

They do this using an optical media which consists of a mixture of nano-particles with many particles lying within the write/read laser spot and the nano-particles have different but adjacent optical emission/absorption frequencies. To prevent oxidation, these nano-particles may be encapsulated in plastic beads and spread across a conventional plastic (or other material) optical disc substrate.

The combination of these nano-particles, in which spectral sensitivities are close to and partly overlapping each other, results in a combined spectral emission profile that is called a “top-hat” fluorescence emission profile, that is, the light emitted by the different nano-particles, combine together to make a pattern like that shown in (a) in the figure below.

With this media, tuning the write laser frequency to match that of one of the nano-particle frequencies creates a spectral hole where the nanoparticle emission can be diminished as shown in (b) in the figure below. These spectral holes can have a depth that depends upon the level of laser energy during the spectral hole writing as shown in the figure. Data can be encoded in both the frequency and depth of these spectral holes.

Optera believes that, short term, 1TB discs are feasible with particulate media and medium term, a thin film single layer write once archival disc with high volume manufacturing costs of $1/10TB ($0.10/TB) is possible before the end of the decade. Longer term (say within a decade or so), these costs could be reduced even further, if this technology were implemented as a volumetric recording technology (perhaps even having 10X lower cost, $0.01/TB).

Projections of new optical storage archiving systems promise 1PB optical cartridges by the 2030’s, compared to the possible 576TB cartridge projections for LTO Gen 14. Optical storage could be a significant contender for the digital archive market.