[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [h5md-user] Strings again / parallel issue
From: |
Pierre de Buyl |
Subject: |
Re: [h5md-user] Strings again / parallel issue |
Date: |
Tue, 4 Feb 2014 19:42:19 +0100 |
User-agent: |
Mutt/1.5.21 (2010-09-15) |
On Tue, Feb 04, 2014 at 01:26:43PM -0500, Peter Colberg wrote:
> On Tue, Feb 04, 2014 at 06:19:20PM +0100, Pierre de Buyl wrote:
> > On Tue, Feb 04, 2014 at 10:38:55AM -0500, Peter Colberg wrote:
> > > Maybe you could use an enumeration for the datasets in question?
> > >
> > > The variable-length string type is currently used with attributes,
> > > which cannot be written in parallel. So I would not expect any
> > > problem for the H5MD specification.
> >
> > Well, the code that I attach (for real now) writes an attribute in
> > parallel. The
> > operation is collective (as it must).
>
> > !
> > ! Create property list for collective dataset write
> > !
> > CALL h5pcreate_f(H5P_DATASET_XFER_F, plist_id, error)
> > CALL h5pset_dxpl_mpio_f(plist_id, H5FD_MPIO_COLLECTIVE_F, error)
>
> > call h5acreate_f(dset_id, 'author', filetype, aspace_id, attr_id, error)
> > if (mpi_rank.eq.0) then
> > call h5awrite_f(attr_id, filetype, author, adims, error)
> > end if
>
> Here the attribute is written by the process on rank 0.
>
> Collective reading and writing is only possible for datasets.
Check the other file, parallel_string_attribute.f90. The call to h5awrite_f is
done on every rank. And it writes the attribute correctly. The program
parallel_vl_string_attribute.f90 does not execute properly.
> > call H5Sselect_hyperslab_f(filespace, H5S_SELECT_SET_F, offset, count,
> > error)
> > !
> > ! Write the dataset collectively.
> > !
> > CALL h5dwrite_f(dset_id, H5T_NATIVE_INTEGER, data, dimsfi, error, &
> > mem_space_id=memspace, file_space_id=filespace, xfer_prp = plist_id)
>
> The above call uses collective I/O, but it is for a dataset.
The call to h5dwrite_f is not related to the attribute issue.
> So far there is no conflict with the current version of H5MD, which
> mandates the vlen string type only for the specified attributes.
>
> For large datasets (scaling with the number of particles) of string
> type that we specify in a later version, fixed-length strings can of
> course be used. But again, if the string values form a small set,
> an enumeration data type is more efficient.
Still, you can't use VL-based data for datasets or attributes in parallel.
> > > Could you describe the data layout of your topology?
> >
> > In another thread if you wish :-)
>
> The description certainly belongs to this thread.
>
> How is the data layed out, in terms of data space and data type?
At some point I wanted <bond_list>: VL Integer[variable] where each item is a VL
Integer of length 2*N_bonds (for pairs) (in the case of time independent data,
[variable] is skipped, indeed). But this will not work because of the parallel
VL issue. <bond_list>: VL Integer[variable][N_bonds_max][2] might be a better
option.
P
- [h5md-user] Strings again / parallel issue, Pierre de Buyl, 2014/02/04
- Re: [h5md-user] Strings again / parallel issue, Peter Colberg, 2014/02/04
- Re: [h5md-user] Strings again / parallel issue, Pierre de Buyl, 2014/02/04
- Re: [h5md-user] Strings again / parallel issue, Peter Colberg, 2014/02/04
- Re: [h5md-user] Strings again / parallel issue,
Pierre de Buyl <=
- Re: [h5md-user] Strings again / parallel issue, Peter Colberg, 2014/02/04
- Re: [h5md-user] Strings again / parallel issue, Peter Colberg, 2014/02/04
- Re: [h5md-user] Strings again / parallel issue, Pierre de Buyl, 2014/02/05
- Re: [h5md-user] Strings again / parallel issue, Peter Colberg, 2014/02/05
- Re: [h5md-user] Strings again / parallel issue, Felix Höfling, 2014/02/06
- Re: [h5md-user] Strings again / parallel issue, Peter Colberg, 2014/02/06
- Re: [h5md-user] Strings again / parallel issue, Pierre de Buyl, 2014/02/06
- Re: [h5md-user] Strings again / parallel issue, Felix Höfling, 2014/02/07
- Re: [h5md-user] Strings again / parallel issue, Pierre de Buyl, 2014/02/07
- Re: [h5md-user] Strings again / parallel issue, Peter Colberg, 2014/02/07