Re: Unionmount: proxying the control port

[Sergiu Ivanov]

Hello,

On Wed, Jul 08, 2009 at 09:39:06PM +0200, Carl Fredrik Hammar wrote:
> On Wed, Jul 08, 2009 at 03:41:51PM +0300, Sergiu Ivanov wrote:
> > Well, the fact that currently unionmount functionality is implemented
> > as additional option of unionfs should not influence the set of
> > use-cases.  I mean that if unionmount is mainly about merging the
> > underlying filesystem and the filesystem of the mountee, we don't
> > really need to modify run-time options of unionmount, regardless of
> > the way it works at the moment.
> 
> Perhaps I should clarify, I meant that it doesn't make sense to forward
> the RPC completely if implemented in unionfs, since unionfs does have
> run-time options that users might want to fiddle with.  That is,
> forwarding should only be done for the --mount option.

Ah, I see your point.  I didn't realize that only the --mount option
could be forwarded.  Unfourtunately, I also realize that such
forwarding again brings about the problem of flawed mountee command
line parsing (via argz_create_sep).
 
> > > > There also are some RPCs which I am not certain about:
> > > > 
> > > > * fsys_getfile: I don't really understand what this one does.
> > > 
> > > This (and file_getfh) enables a client to go back and forth between file
> > > ports and symbolic identifiers unique to the file (for this translator).
> > > Judging from file_getfh's comment this is only used by NFS.
> > 
> > Yeah, I came to a similar conclusion when I looked at file_getfh and
> > fsys.h, but what really embarrasses me is that the file handle from
> > which fsys_getfile extracts the port right is data_t; this tells me
> > nothing about what this file handle is.  You call this ``symbolic
> > identifier'', could you please give some details?
> 
> I don't know the details.  :-(

To see what exactly a ``file handle'' represents, one should look into
hurd/libdiskfs/fhandle.h, where diskfs_fhandle is defined.  The struct
nested in this union contains the following (apparently meaningful)
fields:

* cache_id -- a kind of index into the node cache maintained by the
  libdiskfs-based translator; the translator is supposed to implement
  diskfs_cached_lookup, which should return the node whose cache_id
  field is equal to the supplied value of cache_id;

* gen -- this value should be equal to the st_gen field in the stat
  structure corresponding to the node fetched from the cache
  (i.e. dn_stat.st_gen field); if the values are not equal,
  diskfs_S_fsys_getfile returns ESTALE.

Now the mechanism of conversion handle <-> port is clear: fsys_getfile
extracts the cache index from the supplied handle, looks up the
corresponding node, checks if the values of gen and dn_stat.st_gen are
equal, and returns the port to the found node.  The port is opened on
behalf of the user whose authority is supplied in {n,}uids and
{n,}gids parameters.  file_getfh, OTOH, extracts the values of
cache_id and dn_stat.st_gen from the node it is invoked on, packs
these values into a diskfs_fhandle, and puts the result in its out
arguments.

Note that the handle_len parameter of diskfs_S_fsys_getfile and fh_len
parameter of diskfs_S_file_getfh are used for additional (sanity)
checks only and *must* be equal to sizeof (diskfs_fhandle): the former
function refuses to operate if handle_len != sizeof (diskfs_fhandle),
while the latter allocates more space if fh_len < sizeof
(diskfs_fhandle).

Also note that no other program but nfsd in the Hurd source tree uses
fsys_getfile and file_getfh (and, consequently, file handles).  Also,
the content of a file handle seems pretty artificial, so Fredrik posed
the question whether the concept of a file handle makes much sense at
all and whether nfsd could be implemented without it.
 
> Actually you should study how nfsd makes use of the handle.  In particular
> find out if it assumes that the handle is globally unique, if it does
> then you can to.  Then you could pass it to all file systems and use
> the first one that accepts the handle.

I tried hard to figure out how nfsd uses the handle, but why it is
necessary still evades my comprehension.  Unfourtunately, the part of
the nfsd source code I was looking into was rather poorly documented,
so I cannot state I properly understand why nfsd needs file handles.

However, (as discussed on the IRC), I am able to conclude that the
handle by itself is not globally unique.  Firstly, it most naturally
results from the definition I've described above.  Secondly, nfsd
seems to concoct a globally unique thing by maintaining such a
structure: {filesystem index; file handle} (consider
hurd/nfsd/cache.c, functions fh_hash and lookup_cache_handle).  This
means that polling the filesystems will not work.

> > > The question is which file handle goes to which of the unioned file
> > > systems, this probably can't be determined reliably.  Also you might
> > > not have the authority to obtain the control port of the underlying
> > > file system.  Without this, it can't be implemented.
> > 
> > Hm...  If one doesn't have the authority to obtain the control port of
> > the underlying filesystem, how does have the right to set unionmount
> > on that filesystem at all?  Or do I understand something wrong?
> 
> You can normally set a translator on files you own, no need to have
> a control port to the underlying file system (just to the translator
> you're setting).

Ah, I see.  You are talking about being non-root, being able to set
translators in your home directory, but not being able to get the
control port of the filesystem your home directory is part of, right?
Sorry, I'm generally sluggish in auth matters (because I don't know
them :-( )
 
> > > You'll want to study how extfs/diskfs implements file_getfh before you
> > > reach any final conclusion though.
> > 
> > I'd suppose you mean fsys_getfile ;-) Yeah, libdiskfs is a nice source
> > of inspiration.
> 
> I meant file_getfh, so that you could perhaps gain info on how handles
> are typically formed.  But studying how nfsd actually uses the handles
> is more authoritative since that's the main use-case.

Yeah, your were absolutely right with diskfs_S_file_getfh.  Looking at
this function was my key to understanding how things work.  Thanks for
the suggestion :-)

 > > Also this should be implemented in unionfs as well (if its possible).
> > 
> > Looking at diskfs_S_fsys_getfile, I'm afraid that things are not so
> > simple.  The idea is that both {diskfs,netfs}_S_fsys_getfile get a
> > parameter char * handle.  diskfs_S_fsys_getfile converts this to a
> > const union diskfs_fhandle* .  However libnetfs does not define
> > anything more or less similar to this (I grepped for ``handle''), so I
> > don't know how to treat that parameter.  Perhaps, I could use the same
> > fhandle union as in libdiskfs, what do you think?
> 
> That is probably fine.

Indeed, this should be fine for those libnetfs-based translators which
return ports to their own nodes only.  In this case defining a
structure similar to diskfs_fhandle and maintaining some kind of node
cache will work out.  Unfourtunately, unionfs (as well as a number of
other translator, BTW), gives off ports to the underlying filesystems,
too, so this RPC cannot be properly implemented.

One could think of combining the index of a filesystem and the result
returned by file_getfh when run on a file in this filesystem to obtain
a unique file handle.  However, the fact that the uniqueness should be
assured by only a four-byte number greatly complicates the situation.
Also, due to changing the run-time options or using the stow feature,
the list of filesystems maintained by unionfs can be pretty variable,
rendering it virtually impossible to guarantee that any number
obtained in this way will still make sense at a certain moment in the
future.

That's why I'd rather think of Fredrik's idea about removing the file
handle concept from nfsd at all than of implementing fsys_getfile and
file_getfh in unionfs :-)

> > Also, I cannot find a single libnetfs-based translator in the Hurd
> > source tree that would implement the netfs_S_fsys_getfile in a
> > different way than returning EOPNOTSUPP.
> 
> Depending on the details of file handles, there might be a place for
> a generic implementation in libnetfs eventually.

Well, I did forget to mention that libnetfs implements this RPC in
exactly the same way: returning EOPNOTSUPP.

For the sake of pushing the horizon of my knowledge, I'd like to ask
one more question: is nfs a kind of an interface to nfsd, the latter
being responsible for actually fetching the remote filesystems,
maintaining caches, etc., while nfs's role being to publish this as a
virtual filesystem?  (I'm asking because I was surprised when I found
out that there not only is nfs, but also nfsd.)
 
Regards,
scolobb