HDF5

To use the HDF5 module, add it to your environment. Use:

module spider hdf5 

to see which versions are available and how to load the module and the needed prerequisites.

Example, loading HDF5 version 1.8.17, GCC, OpenMPI

ml GCC/5.4.0-2.26  
ml OpenMPI/1.10.3
ml HDF5/1.8.17

Or one of the other combinations given by 'ml spider HDF5/1.8.17'. Note that you need to load both icc and ifort in addition to other prerequisites listed, despite what 'ml spider' says about only loading modules from one line. Also make sure you load matching prerequisites (i.e. same flavour of MPI etc.)

Example, loading HDF5 version 1.8.17, icc, ifort, CUDA, impi

ml icc/2017.1.132-GCC-5.4.0-2.26
ml ifort/2017.1.132-GCC-5.4.0-2.26
ml CUDA/8.0.44
ml impi/2017.1.132
ml HDF5/1.8.17

You can read more about loading modules on our Accessing software with Lmod page and our Using modules (Lmod) page.

Loading the module should set any needed environmental variables as well as the path. 

Example programs

There are example programs in the examples directory of the HDF5 installation. After loading the module, they can be found here: $EBROOTHDF5/share/hdf5_examples.

Usage

HDF5 is used by adding calls to your program, depending on whether you want to create a new data file, read from an existing data file, or write to an existing data file.

Remember, you must add the HDF header file

#include "hdf5.h"  | C
#include "H5Cpp.h" | C++
USE HDF5           | Fortran

to the header of your program.

File Access Modes

• H5Fcreate accepts H5F_ACC_EXCL or H5F_ACC_TRUNC
• H5Fopen accepts H5F_ACC_RDONLY or H5F_ACC_RDWR
• H5F_ACC_EXCL If the file already exists, H5Fcreatefails. If the file does not exist, it is created and opened with read-write access. (Default)
• H5F_ACC_TRUNC If the file already exists, the file is opened with read-write access, and new data will overwrite any existing data. If the file does not exist, it is created and opened with read-write access.
• H5F_ACC_RDONLY An existing file is opened with read-only access. If the file does not exist, H5Fopen fails. (Default)
• H5F_ACC_RDWR An existing file is opened with read-write access. If the file does not exist, H5Fopen fails.

Creating a file

• Define the file creation property list
• Define the file access property list
• Create the file
• More information here about creating a file, opening an exisiting file, and closing a file.

Examples from HDF homepage

Creating an HDF5 file using property list defaults

file_id = H5Fcreate ("SampleFile.h5", H5F_ACC_EXCL,
    H5P_DEFAULT, H5P_DEFAULT)

Creating an HDF5 file using property lists

fcplist_id = H5Pcreate (H5P_FILE_CREATE)
  <...set desired file creation properties...>
faplist_id = H5Pcreate (H5P_FILE_ACCESS)
  <...set desired file access properties...>
file_id = H5Fcreate ("SampleFile.h5", H5F_ACC_EXCL, fcplist_id, faplist_id)

Opening an HDF5 file (read-only access)

faplist_id = H5Pcreate (H5P_FILE_ACCESS)
status = H5Pset_fapl_stdio (faplist_id)
file_id = H5Fopen ("SampleFile.h5", H5F_ACC_RDONLY, faplist_id)

Closing an HDF5 file

status = H5Fclose (file_id)

Viewing a file with h5dump

Included with the HDF5 distribution is a command-line utility called h5dump. This is a program for inspecting the contents of a HDF5 file. It displays ASCII output formatted according to the HDF5 DDL grammar.

Displaying the content of file.h5:

h5dump SampleFile.h5 

This is how the 'default' file will look, before any datasets or groups have been created, and no data has been written:

    HDF5 "file.h5" {
    GROUP "/" {
    }
    }

You can read more about the program h5dump here.

The HDF5 DDL grammar is described in this document.

File Function Summaries

• Table of general library functions, macros (H5)
• Table of file functions (H5F)
• File creation property list functions (H5P)
• File access property list functions (H5P)
• File driver functions (H5P)

File Property Lists

Additional information regarding file structure and access are passed to H5Fcreateand H5Fopenthrough property list objects. Property lists provide a portable and extensible method of modifying file properties via simple API functions. There are two kinds of file-related property lists:

• File creation property lists
• File access property lists

You can read more about file property lists in the HDF5 User Guide.

Code Examples

The following examples (for version 1.8.X) are taken from the HDF5 User Guide - Code Examples.

• Reading/writing a chunked dataset (Chunking refers to a storage layout where a dataset is partitioned into fixed-size multi-dimensional chunks.) [C] [Fortran]
• Reading/writing a compact dataset [C] [Fortran]
• Reading/writing an external dataset [C] [Fortran]

Compiling

First, you must load the HDF5 module as mentioned at the beginning of this section.

Then, you compile with

• h5cc hdf5_program.c (for C programs)
• h5c++ hdf5_program.cpp (for C++ programs)
• h5fc program.f90 (for Fortran 90 programs)

You can get more help with this by typing

h5cc --help
h5c++ --help
h5fc --help 

Examples, compiling, running

HDF5, version 1.8.17, GCC compiler.

C (h5ex_d_chunk.c - get from download link above)

b-an01 [~]$ ml GCC/6.3.0-2.27  OpenMPI/2.0.2
b-an01 [~]$ ml HDF5/1.8.17
b-an01 [~]$
b-an01 [~]$ h5cc h5ex_d_chunk.c -o h5ex_d_chunk
b-an01 [~]$ ./h5ex_d_chunk
Original Data:
 [   1   1   1   1   1   1   1   1]
 [   1   1   1   1   1   1   1   1]
 [   1   1   1   1   1   1   1   1]
 [   1   1   1   1   1   1   1   1]
 [   1   1   1   1   1   1   1   1]
 [   1   1   1   1   1   1   1   1]

Storage layout for DS1 is: H5D_CHUNKED

Data as written to disk by hyberslabs:
 [   0   1   0   0   1   0   0   1]
 [   1   1   0   1   1   0   1   1]
 [   0   0   0   0   0   0   0   0]
 [   0   1   0   0   1   0   0   1]
 [   1   1   0   1   1   0   1   1]
 [   0   0   0   0   0   0   0   0]

Data as read from disk by hyperslab:
 [   0   1   0   0   0   0   0   1]
 [   0   1   0   1   0   0   1   1]
 [   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0]
 [   0   1   0   1   0   0   1   1]
 [   0   0   0   0   0   0   0   0]
b-an01 [~]$

Fortran 90 (h5ex_d_chunk.f90 - get from download link above)

b-an01 [~]$ ml GCC/6.3.0-2.27  OpenMPI/2.0.2
b-an01 [~]$ ml HDF5/1.8.17 
b-an01 [~]$ 
b-an01 [~]$ h5fc h5ex_d_chunk.f90 -o h5ex_d_chunk
b-an01 [~]$ ./h5ex_d_chunk 

Original Data:
 [  1  1  1  1  1  1  1  1 ]
 [  1  1  1  1  1  1  1  1 ]
 [  1  1  1  1  1  1  1  1 ]
 [  1  1  1  1  1  1  1  1 ]
 [  1  1  1  1  1  1  1  1 ]
 [  1  1  1  1  1  1  1  1 ]

Storage layout for DS1 is: H5D_CHUNKED

Data as written to disk by hyberslabs:
 [  0  1  0  0  1  0  0  1 ]
 [  1  1  0  1  1  0  1  1 ]
 [  0  0  0  0  0  0  0  0 ]
 [  0  1  0  0  1  0  0  1 ]
 [  1  1  0  1  1  0  1  1 ]
 [  0  0  0  0  0  0  0  0 ]

Data as read from disk by hyperslab:
 [  0  1  0  0  0  0  0  1 ]
 [  0  1  0  1  0  0  1  1 ]
 [  0  0  0  0  0  0  0  0 ]
 [  0  0  0  0  0  0  0  0 ]
 [  0  1  0  1  0  0  1  1 ]
 [  0  0  0  0  0  0  0  0 ]
b-an01 [~]$