Sponsored There is no doubt it can be difficult to navigate the various claims made by storage vendors when it comes to performance of their products.
The hype surrounding All Flash Arrays (AFA) has created something of a minefield for users. Headline figures on IOPs and latency can appear fantastic. The first can be reported as being fantastically high and the second as so low as to appear impossibly responsive.
In the AFA space, when it comes to latency there are many factors which can lead to degradation in performance. And those factors are not always present in lab condition testing from which the best numbers are reported. So demonstrating ultra low latency and super high IOPs doesn’t always translate to normal working conditions.
The question is what is being measured? Has latency when operating at high capacity been factored in? And what are other factors that could impact latency of an All Flash Array. Have latency challenges been addressed at the development and design stage? Have solutions to how latency might be impacted through normal use of the solution then been baked into the design.
One obvious factor in latency response times is capacity. Often in AFAs, the higher the capacity the slower the response times and the greater the latency.
Huawei, has taken this into account through detailed long-term research and development of the hardware and software and rigorous testing on its Huawei OceanStor Dorado V3 series all-flash arrays to ensure ultra-low 0.5 ms latency while satisfying capacity loads into the 200+ TB range.
This is achieved through a comprehensive list of technologies and solution led thinking. These include direct controller links, addressing meta data caching issues through memory to avoid issues such as what happens as more users are added and reducing the amount of redundant meta data through highly efficient compression algorithms. The application of dynamic partitioning mechanisms to optimize memory use and optimizing of swap mechanisms all contribute to maintaining low latency even at 80% capacity load.
By adding a 12 Gb/s SAS direct link for the controller enclosure Huawei maintains latency even as higher capacity is achieved through the addition of more enclosures.
A major drag on latency is metadata and the caching of large amounts of metadata generated through granular snapshot, deduplication and ROW to LUN (Logical Unit Numbers) which are common features in AFAs.
Huawei OceanStor Dorado V3 delivers up to 1 TB per engine in cache and by opting instead to use most of its memory for caching the metadata to meet the access requirements when the all-flash arrays are under heavy load. This design ensures high performance and low latency even as capacity utilization reaches the 80% and above mark.
This means directly addressing and overcoming the additional users issue and its impact on latency.
Huawei says: “Metadata is used to describe user data. In a typical all-flash array, most metadata is used in LUN LBA (Logical Block Addressees) mapping and fingerprint tables. This data essentially describes the fingerprint correlation between user data and the LUN LBA. This data is accessed multiple times during the I/O process, which means access performance will significantly impact system performance. The metadata grows as more users are added. Finding ways to more efficiently collate the metadata will allow more to be cached in the space of the memory signature. This is also the precise way to provide users with high performance while maintaining consistently low latency under high capacity usage. Reducing the redundant data, reorganizing the metadata information, distinguishing how the disk and memory layouts are treated, and other methods make for some rather impressive static improvements in how metadata is ingested, processed, stored, and called up.”
Recognising the metadata information means being able to distinguish between different meta data types. By developing compression algorithms to suit the metadata Huawei engineers can save 40% in memory space overhead through optimization.
OceanStor Dorado V3 also applies dynamic partitioning mechanisms. This means when the space in a memory partition is about to run out, the mechanisms on the OceanStor Dorado V3 can add more resources to accommodate new requirements. The opposite is also true: when resources are sitting idle, the system can recover those partition stores.
This avoids having to allocate large amounts of memory for complete planning of pools and LUNs. If users are not utilizing a certain part of the memory, that portion can be dynamically allocated to the metadata for caching.
Developers on the OceanStor Dorado V3 team took up the challenge of hot data by designing new intelligent algorithms to optimize the swap mechanisms.
It says ‘optimization of cache eviction algorithms reduces the CPU overhead needed by the algorithms. On-board mechanisms in the OceanStor Dorado V3 can distinguish between hot and cold data, which means the system can then evict the cold data first and bring cache misses down to a minimum. The system applies theoretical modeling to the cost of evicting the different types of metadata in the system. The modeling classifies and sorts the data. An eviction prioritization strategy is then set based on the calculations to ensure the lowest cost in the eviction process.’ The benefit of the optimizations in the swap mechanisms is one of the secrets behind how the OceanStor Dorado V3 can ensure 0.5 ms latency and high performance under heavy capacity load.
When it comes to writing to disk, here again Huawei took a proactive approach. The array organizes the data to be written to disks and establishes communication mechanisms with the disks. This requires that the arrays and disks be able to interact with each other during production processes
Huawei concludes that there is too much opacity in the test data offered and available to the market.
It says “There is no uniform standard amongst storage vendors in the metrics used to assess their products, which means the results can be craftily worked out by careful selection of parameters. Many vendors spew out performance indicators but the parameters used in the testing are often unclear, which makes forming an accurate assessment of the abundance of offerings available rather difficult. Huawei OceanStor Dorado V3 maintains a stable 0.5 ms latency under 80% load on capacity without having to create a smokescreen, making it the compelling choice in all-around stability in terms of latency.”
SPONSORED BY Huawei
Biting the hand that feeds IT
