Re: cunit [Was: Re: mailutils]

[Sam Roberts]

Quoteing address@hidden, on Thu, Mar 21, 2002 at 03:13:24PM -0500:
> > The *unit test stuff (based on a 5 minute look
> > at the cppunit docs :-)) are fancy asserts. Maybe nice, but
> > certainly not necessary.
> 
> What do you suggest?

assert(); :-)

Or cunit(). I installed it, though, and it looked like an assert()
wrapper. We could upgrade later.

> > To do this in mailutils I would want to
> > 
> > - fork a server that I will communicate with via a pipe
> 
> Possible, most of the daemon can be started in inetd mode
> so if we make the setsockopt() a non fatal error in testing
> mode they can work via a pipe.

It also fails if it can't setgid(), and it chroot()s to /. I've
taken a few half-hearted attempts to change this, but didn't
want to hack it up without a little discussion.

> > - run negative low-level tests, i.e. hard code a long pop3 dialog
> >   with obscenely long commands to try and trigger buffer overflows
> > - open mailboxes to talk to the server, and call our APIs with
> >   both reasonable and totally unreasonable arguments
> > 
> > This would be easy, we just need two things:
> > 
> > - the ability to tell a server to run either chrooted, or to only access
> >   config and mail spools in the test tree
> 
> would not the option --maildir do the trick?

Possibly.

I just tried this:

# cp /usr/spool/mail/sam sam
# ./pop3d --inetd --maildir=`pwd`
+OK POP3 Ready <address@hidden>
user sam
+OK
pass secret
-ERR [IN-USE] Mailbox in use
quit
+OK
# echo ./pop3d --inetd --maildir=`pwd`
/pop3d --inetd --maildir=/home/sam/p/gnu/mailutils/pop3d
#

So even as root it doesn't work out of the box. But it shouldn't
be hard.

Oh, and I found a bug, too, in my locker_unlock() when you ctrl-C
it... already we are seeing results! ;-)

> > 
> > - open a mailbox on arbitrary file descriptors (can we do this by
> >   setting the stream after creating one?)
> 
> probably, we could create an arbitrary stream_t object that sends
> garbage or good data intermix with garbage.

I was thinking for nasty testing on the server you want to fork
one and do writes() and reads() to try and kill it with garbage.

To test the mailboxes it would be nice to do

mailbox_create("pop://user:pass", &mbox);
stream_t s;
stream_create_popen("pop3d --inetd --maildir=`pwd`", &s);
mailbox_set_stream(mbox, s);
mailbox_open(); // have it just just the stream, rather than opening
  // a tcp connection
mailbox_stat();
..
.

Just start coding, asserting that the answer is what you expect.

Actually, we don't even need this. We can for pop3d on a unique
tcp port:

spawn("pop3d --inetd --maildir=/tmp/mailutils --port=5678");

mailbox_open("pop://sam:address@hidden:5678");
...
..
..

The biggest thing for me is having to run the tests as root. We
shouldn't have to.

> > Also, memory leak testing is usually naive, it doesn't test the
> > error paths, particularly whether memory is leaked in the cleanup
> > paths triggered by a failed alloc. There are too many people running
> > Insure++ saying "look, we don't have memory leaks",  but you take their
> > code, start failing mallocs, and it usually doesn't leak, it explodes.
> 
> 
> That's a good point.  The library itself try not to use things like xmalloc()
> which simply exit() when malloc() failed but rather throw the error up and 
> let the policy decision be made above.

> 200% agree.  I'm ready to spend the few cycles
> if you can point to me at an interesting framework for tests
> [cunit] ?

http://people.codefactory.se/~spotty/cUnit/

Gotta run, I'm supposed to be eating dinner right now!
Sam

-- 
Sam Roberts <address@hidden> (Vivez sans temps mort!)