Skip to content

Commit

Permalink
Update hdf5_1_12.md
Browse files Browse the repository at this point in the history
  • Loading branch information
bmribler authored Mar 14, 2024
1 parent 14fd0db commit 6fb7366
Showing 1 changed file with 7 additions and 28 deletions.
35 changes: 7 additions & 28 deletions documentation/hdf5-docs/release_specifics/hdf5_1_12.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,34 +4,13 @@ redirect_from:
- display/HDF5/New+Features+in+HDF5+Release+1.12
---

This release includes changes in the HDF5 storage format. PLEASE NOTE that HDF5-1.10 and earlier releases cannot read files created with the new features described below that are marked with a *.
Migrating from HDF5 1.10 to HDF5 1.12

HDF5 1.12 introduces several new features in the HDF5 library:
New Features in HDF5 Release 1.12

Virtual Object Layer (VOL) *
Hyperslab Performance Improvement
Update to References *

H5Sencode / H5Sdecode Format Change - RFC
Update to References
Update to Selections
Virtual Object Layer (VOL) (RFC) *
See the Virtual Object Layer page for more information.

The Virtual Object Layer (VOL) is an abstraction layer within the HDF5 library that enables different methods for accessing data and objects that conform to the HDF5 data model. The VOL intercepts all HDF5 API calls that potentially modify data on disk and forwards those calls to a plugin "object driver". The data on disk can be a different format than the HDF5 format.



The plugins can actually store the objects in variety of ways. A plugin could, for example, have objects be distributed remotely over different platforms, provide a raw mapping of the model to the file system, or even store the data in other file formats (like native netCDF or HDF4 format). The user still gets the same data model where access is done to a single HDF5 “container”; however the plugin object driver translates from what the user sees to how the data is actually stored. Having this abstraction layer maintains the object model of HDF5 and allows better usage of new object storage file systems that are targeted for Exascale systems.

Hyperslab Performance Improvements
In 1.12.0 the hyperslab selection code was optimized to achieve better performance. In general, performance improved by an order of a magnitude. In the case of reading a regular selection from a 20 GB dataset into a one dimensional array, performance improved by a factor of 6000. If you are interested in the benchmark we ran, please see issue HDFFV-10930 by logging into jira.hdfgroup.org with your hdfgroup.org login.

Update to References (RFC) *
See the Update to References page for details on the changes in HDF5-1.12.

HDF5 references were extended to support attributes, as well as object and dataset selections that reside in another HDF5 file. In order to support these features several functions were introduced:

Create (H5R_CREATE*) functions were added for each reference type: object, dataset region and attribute.
A function was added to release a reference (H5R_DESTROY). This is required because a region reference no longer modifies the original file.
Functions were added to query references (H5R_GET*).
Other functions were added to simplify or clarify the API.
Update to Selections
Several new H5S APIs were introduced to allow a user to more flexibly operate on two hyperslab selections. See Update to Selections for more details.
Virtual Object Layer
Software Changes from Release to Release for HDF5 1.12

0 comments on commit 6fb7366

Please sign in to comment.