Storage system suppliers showcase progressive options to maintain tempo with quicker accelerators.
MLCommons® introduced outcomes for its industry-standard MLPerf® Storage v1.0 benchmark suite, which is designed to measure the efficiency of storage methods for machine studying (ML) workloads in an architecture-neutral, consultant, and reproducible method. The outcomes present that as accelerator expertise has superior and datasets proceed to extend in measurement, ML system suppliers should make sure that their storage options sustain with the compute wants. It is a time of fast change in ML methods, the place progress in a single expertise space drives new calls for in different areas. Excessive-performance AI coaching now requires storage methods which might be each large-scale and high-speed, lest entry to saved information turns into the bottleneck in all the system. With the v1.0 launch of MLPerf Storage benchmark outcomes, it’s clear that storage system suppliers are innovating to satisfy that problem.
Model 1.0 storage benchmark breaks new floor
The MLPerf Storage benchmark is the primary and solely open, clear benchmark to measure storage efficiency in a various set of ML coaching eventualities. It emulates the storage calls for throughout a number of eventualities and system configurations masking a spread of accelerators, fashions, and workloads. By simulating the accelerators’ “assume time” the benchmark can generate correct storage patterns with out the necessity to run the precise coaching, making it extra accessible to all. The benchmark focuses the check on a given storage system’s skill to maintain tempo, because it requires the simulated accelerators to take care of a required degree of utilization.
Three fashions are included within the benchmark to make sure various patterns of AI coaching are examined: 3D-UNet, Resnet50, and CosmoFlow. These workloads provide quite a lot of pattern sizes, starting from a whole bunch of megabytes to a whole bunch of kilobytes, in addition to wide-ranging simulated “assume occasions” from just a few milliseconds to some hundred milliseconds.
The benchmark emulates NVIDIA A100 and H100 fashions as representatives of the at the moment accessible accelerator applied sciences. The H100 accelerator reduces the per-batch computation time for the 3D-UNet workload by 76% in comparison with the sooner V100 accelerator within the v0.5 spherical, turning what was sometimes a bandwidth-sensitive workload into way more of a latency-sensitive workload.
As well as, MLPerf Storage v1.0 consists of assist for distributed coaching. Distributed coaching is a vital state of affairs for the benchmark as a result of it represents a typical real-world follow for quicker coaching of fashions with giant datasets, and it presents particular challenges for a storage system not solely in delivering larger throughput but in addition in serving a number of coaching nodes concurrently.
V1.0 benchmark outcomes present efficiency enchancment in storage expertise for ML methods
The broad scope of workloads submitted to the benchmark replicate the wide selection and variety of various storage methods and architectures. That is testomony to how vital ML workloads are to all forms of storage options, and demonstrates the energetic innovation occurring on this house.
“The MLPerf Storage v1.0 outcomes exhibit a renewal in storage expertise design,” mentioned Oana Balmau, MLPerf Storage working group co-chair. “For the time being, there doesn’t look like a consensus ‘better of breed’ technical structure for storage in ML methods: the submissions we obtained for the v1.0 benchmark took a variety of distinctive and artistic approaches to offering high-speed, high-scale storage.”
The leads to the distributed coaching state of affairs present the fragile steadiness wanted between the variety of hosts, the variety of simulated accelerators per host, and the storage system in an effort to serve all accelerators on the required utilization. Including extra nodes and accelerators to serve ever-larger coaching datasets will increase the throughput calls for. Distributed coaching provides one other twist, as a result of traditionally completely different applied sciences – with completely different throughputs and latencies – have been used for transferring information inside a node and between nodes. The utmost variety of accelerators a single node can assist is probably not restricted by the node’s personal {hardware} however as an alternative by the power to maneuver sufficient information rapidly to that node in a distributed surroundings (as much as 2.7 GiB/s per emulated accelerator). Storage system architects now have few design tradeoffs accessible to them: the methods should be high-throughput and low-latency, to maintain a large-scale AI coaching system operating at peak load.
“As we anticipated, the brand new, quicker accelerator {hardware} considerably raised the bar for storage, making it clear that storage entry efficiency has develop into a gating issue for total coaching pace,” mentioned Curtis Anderson, MLPerf Storage working group co-chair. “To stop costly accelerators from sitting idle, system architects are transferring to the quickest storage they’ll procure – and storage suppliers are innovating in response.”
MLPerf Storage v1.0
The MLPerf Storage benchmark was created via a collaborative engineering course of throughout greater than a dozen main storage answer suppliers and tutorial analysis teams. The open-source and peer-reviewed benchmark suite provides a degree taking part in subject for competitors that drives innovation, efficiency, and vitality effectivity for all the {industry}. It additionally supplies essential technical info for purchasers who’re procuring and tuning AI coaching methods.
The v1.0 benchmark outcomes, from a broad set of expertise suppliers, exhibit the {industry}’s recognition of the significance of high-performance storage options. MLPerf Storage v1.0 consists of over 100 efficiency outcomes from 13 submitting organizations: DDN, Hammerspace, Hewlett Packard Enterprise, Huawei, IEIT SYSTEMS, Juicedata, Lightbits Labs, MangoBoost, Nutanix, Simplyblock, Volumez, WEKA, and YanRong Tech.
“We’re excited to see so many storage suppliers, each giant and small, take part within the first-of-its-kind v1.0 Storage benchmark,” mentioned David Kanter, Head of MLPerf at MLCommons. “It reveals each that the {industry} is recognizing the necessity to maintain innovating in storage applied sciences to maintain tempo with the remainder of the AI expertise stack, and likewise that the power to measure the efficiency of these applied sciences is essential to the profitable deployment of ML coaching methods. As a trusted supplier of open, truthful, and clear benchmarks, MLCommons ensures that expertise suppliers know the efficiency goal they should meet, and customers can procure and tune ML methods to maximise their utilization – and in the end their return on funding.”
View the Outcomes
To view the outcomes for MLPerf Storage v1.0, please go to the Storage benchmark results.
Join the free insideAI Information newsletter.
Be part of us on Twitter: https://twitter.com/InsideBigData1
Be part of us on LinkedIn: https://www.linkedin.com/company/insideainews/
Be part of us on Fb: https://www.facebook.com/insideAINEWSNOW
