mailbox (was Re: intro)

[xystrus]

On Wed, Mar 13, 2002 at 08:56:58AM -0500, Alain Magloire wrote:
> > Actually I've been poking through the libmailbox code a little bit,
> > and I think I would like to play with that for a while.  On the whole
> > it looks to me like it's pretty well written.  Of particular interest
> 
> If mail handling was that simple 8-).
> No, you are looking at the wrong code, libmailbox was our first
> version way back, it is obsolete now.  The code is in mailbox/*
> all the added made it, .. a "little" more complex 8-).
> This is what I meant by things getting too complex, but I understand
> why the complexity and thought we could have done a better job
> handling this .. but that's allright, next time around with the experience 8)

Ok, now I understand your comment, as well as your mentioning you wanted
to separate the functionality of the lower APIs from the POP and IMAP
stuff.  FWIW, I agree with you...  I think you were on the right track
with libmailbox.  It's missing a lot of stuff, and there's no hint of
signal handling in it at all, but as far as mailbox access I think
the API you defined is perfect.  All the other stuff should be built
ON TOP of that, i.e. handling MIME is not specifically a part of
mailbox access...  I think it would be best to have the API you 
originally defined for accessing mail folders, and then build seperate
APIs that sit on top of that API for the other stuff.  Though there may
be issues I'm overlooking, and hopefully the list will point those out
to me.

> > to me is what effect switching to MMIO would have on performance and
> > memory utilization, for both small mailboxes and very large mailboxes.
> > I may play with this, and then report my results back here, if there's
> > interest.  In theory, MMIO should have a noticable positive impact on
> > performance, at least in the case of machines with plenty of RAM.  
> 
> Is MMIO, Memory Map files? if yes we do that for Read-only files.

Yes.  MMIO = Memory-Mapped I/O.

> But it turns out that the performance gain is relatively small
> for big files(lots of page fault) and most of the time the spool
> is on remote machine access across NFS.

Right... you do need to have a lot of memory for the gains to be
noticeable.  It still shouldn't be slower, unless you really have
way too little memory in your server.  If that's the case, BUY SOME
RAM!  It's dirt cheap.

Somebody mentioned having a configure option for disabling MMIO later
in this thread, and I'd agree with that.  But (without having done
any empyrical testing) I suspect that for most cases, MMIO will be
faster, and I think the code is easier to write too, so it should
be the default.

> > I'm also interested in the mbox_expunge() function... I haven't
> > gotten around to looking closely at it, but I did notice a comment
> > that claims it breaks splendidly.  Can someone comment on how it
> > breaks?  Is this still a problem?  I've also already written some
> > functions that more-or-less implement expunging messages from mbox
> > folders using MMIO, so it should be reasonably easy to convert what
> > I have into a form that makes sense for libmailbox...
> 
> Interresting, it is a lot of moving around.  Did you notice
> any gains for large mailbox, with random delete messages?

Honestly, I didn't do a lot of comparison testing.  At the time, I
was more interested in just figuring out how to do it than comparing
performance against non-MMIO methods.  As I mentioned, MMIO is fairly
new to me...  I'm also not sure about portability issues, but since
you're already using it, it would seem that's not a problem...

> > It also seems that support for the vaunted Maildir and MH folders
> > isn't finished.  Is someone actively working on these?  I've been
> > meaning to learn more about how Maildir is implemented anyway, so I
> > thought I might have a look at finishing that up.
> 
> MH was done by sergey.
> I do not think anyone is working on maildir.  To do maildir
> you will have to implement must functions mark as "stub"
> in mailbox.c in a file mbx_maildir.c file, see mbx_pop.c,
> mbx_mh.c, mbx_mbox.c for some complex examples.
> 
> > Thoughts?
> 
> Not easy, and one shot of tequilla will not be enough 8)

Well last night was mudslides... ;-)