Re: The sad state of locking -> it almost works

[Sam Roberts]

Bonsoir,

Quoteing address@hidden, on Thu, Apr 11, 2002 at 01:06:52PM +0300:
> This is possible, but consider this:
> 
>  1. Default Linux /bin/mail uses flock to lock mailbox before writing
>     to it.
>  2. Widely-used (on GNU/Linux at least) mail.local uses both dot-lock
>     and kernel-level locking via flock.
>  3. Procmail (again widely used on GNU/Linux) creates dot-lock, but
>     uses flock before writing to the mailbox (see locking.c,
>     mailfold.c).
>  4. Mutt: first applies kernel locking, then creates dot-lock
>     if requested (see mx.c).

Sure, with kernel locking being fcntl, then flock and all of them
ifdefed base on a USE_ compile time option.

> Seems ok. My proposals are to follow the practice shown by procmail,
> mutt and others, that is: use dot-locking and apply kernel-locking
> to the mailbox right before writing to it (of course, this applies
> only to unix-style mailboxes). 

Ok, we do the same as mutt, then? That's what I've been working
towards, I didn't realize that it was doing all 3 (potentially) in sequence,
I should have checked the configure flags.

> > We agreed on adding the read/write lock flag back to locker_lock().
> > 
> >   - kernel locking can use it for rd/wr locks
> > 
> >   - utilities that wish can set a NO_LOCK_ON_READ flag (or it can
> >     be set in --lock-flags), it will do what it sounds like. Read
> >     locks will return sucessfully without actually locking.
> 
> OK.

> Both, I believe. This happens when a mailbox to be read resides in a
> non-writable directory and is non-writable itself. But then, there is
> hardly a reason to lock it, is there?

> To summarize:
> 
> 1. Introducing NO_LOCK_ON_READ solves problem with imap4d.
> 2. Problem with mail on systems whose mailspool is not world-writable
>    may be solved using setuid/setgid dotlock.

I'd say we don't need to support flock(), at least now. Ok?
So, the algorithm starts to look like

Lock (need_write: bool)
begin
 if( no_rdlock && need_write == false )
   goto done:

 if( file not writeable by us)
   goto done;

   // or should this be considered the callers error? This means
   // they a) didn't set no_rdlock or b) are trying to wrlock an
   // unwriteable file
     
 if(use_fcntl)
   fcntl( ...., need_write ? F_WRLCK : F_RDLCK)

 if(use_dot)
   if( we have write access to create dotlock file)
      create it directly; // for setgid() servers
   else
      create it with external utility;

done:
 increment lockcount

end

As an aside, it seems to me we will have to track rd and wr lock counts
seperately, so recursive locks work correctly.

The default should be "use_fcntl true AND use_dotlock TRUE
and no_rdlock true".

Ok for all?

Any comments on the algorithm? Is that what we're going towards?

> However, usual /bin/mail works without creating dotlock files, and
> works quite reliably. So, consider my proposition about
> compulsory kernel locking.

Hm. Works quite well if you're not using NFS, so compulsory in the sense
of "if it's configured to be used, it MUST suceed" I agree with, but
I think it should be possible to configure it off.


> > The UW-IMAP documentation talks about locking at length, and you can
> > read what they have to say here:
> > 
> >   http://www.washington.edu/imap/documentation/locking.txt.html
> 
> Thanks for the link. This is really very useful information. By the way,
> it confirms my viewpoint about kernel locking, to wit (page 14 of the
> document):
> 
> "      (2) c-client applies a shared flock() to the mail file whenever"
> " it reads from the mail file, and an exclusive flock() whenever it"
> " writes to the mail file.  This lock is freed as soon as it finishes"
> " reading.  The purpose of this lock is to prevent against unfavorable"
> " interactions with mail delivery.
>  [...]

" FLOCK() BUGS
" 
" flock() advertises that it permits upgrading of shared locks to
" exclusive and downgrading of exclusive locks to shared, but it does so
" by releasing the former lock and then trying to acquire the new lock.
" This creates a window of vulnerability in which another process can
" grab the exclusive lock.  Therefore, this capability is not useful,
" although many programmers have been deluded by incautious reading of
" the flock() man page to believe otherwise.  This problem can be
" programmed around, once the programmer is aware of it.
" 
" flock() always returns as if it succeeded on NFS files, when in
" fact it is a no-op.  There is no way around this.
" 
" Leaving aside these two problems, flock() works remarkably well,
" and has shown itself to be robust and trustworthy.
" 

lock (1) is dotlocking, and to complete the quote: ;->


  " Mail delivery daemons use lock (1), (2), or both.  Lock (1) works
  " over NFS;
> "         lock (2) is the only one that works on sites that protect"
> " /usr/spool/mail against unprivileged file creation.  Prudent mail"
> " delivery daemons use both forms of locking, and of course so does"
> 
> Opinions?

We should be prudent.

And that last statement is only true if you don't use a privileged
lock utility.

Anyhow, there's a bunch of tradeoffs between the two ways, just as
there are between mailbox formats.

What is important to me is:

- you can choose dot, kernel, or both

- what you ask for, you get.

- when you want dotlocking, you can get a dotlock with an external
  prvivileged utility if necessary

Sam

-- 
Sam Roberts <address@hidden> (Vivez sans temps mort!)

muttlock
Description: Text document