h5md-user
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[h5md-user] Dataset layouts


From: Peter Colberg
Subject: [h5md-user] Dataset layouts
Date: Tue, 9 Jun 2015 16:15:06 -0400
User-agent: Mutt/1.5.23 (2014-03-12)

Hi all,

While working on a truly large-scale scientific application using
the HDF5 MPI I/O driver, I noticed it would be useful to mandate that
“meta-datasets” are always stored with compact layout. This would
apply to proposal 100 (scalar step/time), and `box/edges` for the
fixed-geometry case.

When using parallel HDF5, most C API calls are collective, which means
that a function must be called by all processes in the communicator
used to access a file. Compact datasets are metadata and therefore
must be written/read collectively. This goes nicely with the other
metadata stored in attributes.

In similar spirit, the specification states that time-dependent
datasets have an infinite first dimension. I suggest we specify
explicitly that these datasets must have a chunked layout.

Regards,
Peter



reply via email to

[Prev in Thread] Current Thread [Next in Thread]