How cold storage can solve the growing data problem
Professional sports are currently a hotbed for next-generation technology and data. Over the seasons, hours and hours of digital content are created. From tracking player stats and performance analysis, to video footage capturing every element of the action in every match, from multiple camera angles, in stadiums around the world. That’s a lot of data and everything has to be stored somewhere.
About the Author
Davide Villa, Director of Business Development for EMEAI, Western Digital.
In order to deliver data-rich minute-by-minute files while still capturing the live action, data management teams must decide where to store each hot, hot, or cold data, depending on how fast and how often. to which they must access it.
But that’s not just a problem for the sports industry. Experts estimate that global data flows are increasing by around 30% per year, potentially generating 175 ZB by 2025. While not all of this data needs to be analyzed immediately, its storage is essential, and this is where cold storage comes into play.
The rise of cold rooms
Cold storage is used to hold data that is not actively in use. This data can be stored in archives, otherwise known as “cold” storage. These are less expensive and infrequently accessed storage tiers, as opposed to their live counterparts, of “hot” production data like financial transactions, which must be accessible immediately.
And it’s a storage segment that won’t go away anytime soon, according to industry analysts 60% or more of stored data can be archived or stored in cooler storage tiers until it’s needed. .
As the world generates and stores more archival data than ever before, cold storage is becoming the fastest growing segment in the industry. As more and more data is stored, cloud providers are reinventing their architectures with accessible archives to keep pace and ensure efficient management.
The benefits of cold
With data rising and reaching the age of the zettabyte; the more data stored, the more it costs. The largest amounts of data are often unstructured or semi-structured data, such as video footage, genomics, or data used to train machine learning and AI. Much of this can be stored in cold secondary storage, which is much cheaper than hot primary storage. For data that is not actively needed although it is part of an active process, storing it in cooler storage pools at a lower cost might be the solution.
However, the biggest consideration when using cold storage will be how often you need to access the data or how easily you want it to be available. Today’s cloud storage SLAs are structured around how often data is accessed and how long a customer is willing to wait to retrieve that data. Data stored in a cooler tier can take five to 12 hours to access cloud providers, while data stored in warmer tiers is available immediately but comes at a cost.
Besides the obvious cost and affordability considerations, the third factor for the end user becomes psychological. It almost goes against human nature to delete anything in case you need it at some point and you never know what data will be valuable later.
What are the current options?
Until recently, most secondary cold storage was contained on tapes or hard drives (HDDs), with hot data being moved to solid state drives (SSDs). However, according to Horison Information Strategies, archival data could reach 80% or more of all data captured by 2025, making it by far the largest and most dynamic storage class, presenting the next big one. storage challenge. In addition, the value of data is generally related to the ability to access and use it. In other words, data accessibility increases the value of data.
While tape storage is cheaper than hard drives, it also has higher data access latency, making it an option only for very cold storage. Hard drives are evolving into next-generation drive technologies and platforms to improve both the cost of ownership and the accessibility of active archiving solutions. Recent advancements in HDD technology include new data placement technologies such as zoning, higher area densities, mechanical innovations, intelligent data storage, and new material innovations.
How will the cold room evolve?
The hyperscalers and digital content creators who host the largest data pools are looking for the most cost-effective ways to store their ever-growing amount of data. To meet the demand, new levels of cold storage are emerging and IT organizations are focusing on reinventing archive storage architectures to prepare.
With the time needed to store longer-term data spanning more than a century, scalable cold storage solutions to stand the test of time will be essential. To ensure their sustainability, innovations such as DNA, optics and even underwater freezing are being developed.
The recent creation of the DNA Data Storage Alliance is one of many movements aimed at advancing the field of cold storage. Due to its high density, DNA has the ability to pack large amounts of information in a small space and can exist for thousands of years, making it an attractive medium for archival storage.
With the era of Zettabyte data creating challenges ranging from sustainability to accessibility, cold storage is poised to prove essential to preserving data affordably and for longevity. Therefore, continuous innovation is needed to create long-term data storage solutions that make valuable data accessible both in the short term and for generations to come.