Contributors to this blog include Matias Bjørling, Jorge Campello De Souza, Dave Landsman, Damien Le Moal, and Ted Marena.
We at Western Digital are very concerned about how to architect data infrastructure solutions for zettabyte scale. The demands of applications from IoT, automotive, video creation and surveillance mean that data center systems of this capacity will be a requirement in the not too distant future.
We’ve been working on technologies that create greater efficiencies for massive datasets through the Zoned Storage initiative. Zoned Storage is a framework for intelligently placing data on a device, and is an open-source, standards-based initiative to enable data centers to scale efficiently for the zettabyte storage capacity era.
A set of standards make up Zoned Storage, ZBC (Zoned Block Commands) and ZAC (Zoned ATA Command Set) for SMR (Shingled Magnetic Recording) HDDs and ZNS (Zoned Namespaces) for NVMe™ SSDs. The unifying zoned block interface for both HDDs and SSDs enables software developers and data center architects to realize the promise of Zoned Storage (capacity, costs, and endurance).
In this blog we’d like to focus on ZNS because there are significant milestones which were recently achieved that are particularly of interest for data center architects and developers.
NVMe Specification Ratification – ZNS Is Official
The Zoned Namespace (ZNS) Command Set specification has been ratified by the NVM Express consortium. The specification is available for download under the Developers -> NVMe Specification section of the www.nvmexpress.org public web site, as an NVM Express 1.4 Ratified TP.
With an approved standard, ZNS-based NVMe SSDs are poised to become an integral part of the Zoned Storage device ecosystem, complementing SMR HDDs. By enabling the sequential zoned storage model, ZNS allows the host and the SSD to coordinate data placement onto the SSD, providing higher write endurance and improved I/O access latencies, while enabling technologies such as QLC NAND to proliferate.
Software Upstreaming – Software Development Goes Public
Zoned Storage initial support in Linux® was introduced with the kernel version 4.10. Later kernel versions extended Linux zoned block device interface with new features and added support to various kernel components such as device-mapper drivers. We’ve been working with the open source community to integrate ZNS support in Linux to ensure that NVMe ZNS devices are compatible with the Linux kernel zoned block device interface. The Linux kernel modifications for ZNS have recently been publicly released on the developer’s mailing lists. These changes are expected to be accepted for the next Linux kernel version.
Enabling ZNS support in the Linux kernel is the first step. Modifications to well-known user applications and tools, such as RocksDB, Ceph, and the Flexible IO Tester (fio) performance benchmark tool, together with the new libzbd user-space library, are also being released.
To see further details on the software support, visit https://zonedstorage.io
Adoption and Ecosystem
The ZNS ecosystem is growing rapidly. Of course, Western Digital has been committed to Zoned Storage and ZNS, but many other organizations are now adopting this new standard, including public and private cloud vendors, all flash-array vendors, solid-state device vendors, and test and validation tool suppliers.
Even as ZNS Command Set specification introduces a new zoned storage block interface for SSDs, much of the software changes required to adopt the model is already mature due to the existing SMR HDD software eco-system, accelerating the adoption of ZNS SSDs. With a small set of changes to the software stack, users of host-managed SMR HDDs can deploy ZNS SSDs into their data-centers, and new adopters can take advantage of the existing software eco-system. Furthermore, they can utilize the existing tools to accelerate support in their applications.
The unifying zoned block interface for both HDDs and SSDs enables software developers to support a single interface, accelerating storage deployments, and ultimately taking advantage of the benefits of Zoned Storage (capacity, costs, and endurance).
Where Data Infrastructure is Headed
As data infrastructure is rapidly changing, the momentum is toward open, purpose-built, scalable solutions. NVMe is a technology that will be ubiquitous in data centers moving forward, and ZNS will be key in helping scale storage needs.
To learn more about ZNS and other next-generation storage solutions, join the Storage Solutions Meetup Group and attend the initial event on July 21st. Info here.
Forward-Looking StatementsCertain blog and other posts on this website may contain forward-looking statements, including statements relating to expectations for our product portfolio, the market for our products, product development efforts, and the capacities, capabilities and applications of our products. These forward-looking statements are subject to risks and uncertainties that could cause actual results to differ materially from those expressed in the forward-looking statements, including development challenges or delays, supply chain and logistics issues, changes in markets, demand, global economic conditions and other risks and uncertainties listed in Western Digital Corporation’s most recent quarterly and annual reports filed with the Securities and Exchange Commission, to which your attention is directed. Readers are cautioned not to place undue reliance on these forward-looking statements and we undertake no obligation to update these forward-looking statements to reflect subsequent events or circumstances.