Re: [h5md-user] Dataset layouts

[Pierre de Buyl]

On Tue, Jun 09, 2015 at 04:41:46PM -0400, Peter Colberg wrote:
> Hi Pierre,
> 
> On Tue, Jun 09, 2015 at 10:30:31PM +0200, Pierre de Buyl wrote:
> > I understand the motivation. Did you notice a difference in performance or 
> > is
> > there some other motivation?
> 
> It is not related to performance at all.
> 
> Non-compact and compact datasets have to be handled differently for
> writing and reading. A non-compact dataset is only written/read by one
> process, while a compact dataset is written/read by all processes. 
> 
> When writing/reading a compact dataset from only one process, the
> program will hang. When writing a non-compact dataset from all
> processes, the program will block for a finite period due to
> write conflicts.
> 
> The compact layout is preferable over the non-compact layout for
> efficiency reasons (and being equivalent to attributes), hence I
> propose to mandate the compact layout.

I see no problem with that. I'll need to convince Andrew Colette for the
convenience, though :-)
https://groups.google.com/d/topic/h5py/kbhrO-XhFnE/discussion

P