Re: [h5md-user] Dataset layouts

[Felix Höfling]

Am 15.06.2015, 14:53 Uhr, schrieb Pierre de Buyl  
<address@hidden>:

> On Wed, Jun 10, 2015 at 03:31:34PM +0200, Pierre de Buyl wrote:
> > On Tue, Jun 09, 2015 at 04:41:46PM -0400, Peter Colberg wrote:
> > > The compact layout is preferable over the non-compact layout for
> > > efficiency reasons (and being equivalent to attributes), hence I
> > > propose to mandate the compact layout.
> >
> > I see no problem with that. I'll need to convince Andrew Colette for the
> > convenience, though :-)
> > https://groups.google.com/d/topic/h5py/kbhrO-XhFnE/discussion
>
> Afterthought: as reading the dataset will not be impacted by its  
> compactness,
> does it make sense to mandate it?

Peter, did I understand correctly: parallel reading of a dataset must take  
into account whether the dataset is compact or not, otherwise the data are  
inconsistent between the MPI processes? (Actually, such a mis-use of the  
HDF5 library should raise an exception in my opinion.)

With respect to writing, I don't see any need to require compactness. The  
writer application "knows" whether it uses the MPI interface or not and  
can act accordingly. Second, h5py does a great job in writing H5MD files  
so far. I would not like to break this kind of support by making  
compactness mandatory.

For reading, on the other hand, the implementation of a reader has to be  
simple for the sake of robustness. Querying the storage layout before  
reading may be one complication that can be avoided by specifying the  
layout. (This reminds me of an endless discussion about the string type.)

Felix