Re: [Sks-devel] sks (fast)build memory/cache problem

[Jeffrey Johnson]

On Jul 2, 2012, at 11:05 AM, Kristian Fiskerstrand <address@hidden> wrote:

> On 2012-07-01 04:29, Stephan Beyer wrote:
>> Hi,
>> 
>> On 30.06.2012 13:31, Stephan Beyer wrote:
>>> The DB is already building based on my last keydump.
>> No, it was not. The process got stuck after a few minutes or seconds.
>> In fact, neither build nor fastbuild works.
>> I already "hg bisect"ed the sources and it did not work with any
>> version that compiles on my system.
>> 
>> What happens? sks hangs (deadlocks) at futex(), says strace.
>> Interestingly, it seems to depend on the cache size:
>> 
> 
> I'm still not able to reproduce error in my own setup, but looking for
> various causes I came across an interesting thread at [0] that seem to
> indicate that the filesystem blocksize can be a factor. I'm using 4k
> blocksize on ext3 myself, does anyone having problems happen to use a
> smaller blocksize on the FS?
> 
> 
> [0] https://bugzilla.redhat.com/show_bug.cgi?id=680508
> 

Um there's ancient hysteria and a private agenda at that bug report.

The issue was never ever reproduced with a known
mechanism, and (the private agenda) is a repeat
of an older issue from 2008 that likely has just
been fixed as part of development.

I spent a couple weeks attempting reproducers for
kernel fs guys at the time, never could reproduce.

Meanwhile there is another failure case that needs
to be controlled for
        stale futexes
If a program exits with locked futures, you WILL
have to reboot to clear the stale futex.

I see/hear 1-2 cases of "stale futex" symptoms every
year (most issues are something else, like dueling
rpm versions, etc).

There is a "robust futex" implementation that permits
a different thread/process to clear a futex by setting
a flag which can be added to Berkeley DB. This is
_NOT_ the right patch for "system" Berkeley DB because
the lock is there for a reason, and any other process
that clears the lock is also responsible for whatever
statefulness cleanup the lock was protecting, and this
isn't a generally solvable problem.

But SKS+BDB internal would be more robust in the
face of mysterious "stale futexes".

I've got the 6-10 line patch to BDB+PTHREADS around
somewhere: its just setting a flag when the mutex
is created/acquired and a check for a return code,
and then clearing the "stale futex" when encountered).

73 de Jeff
> -- 
> ----------------------------
> Kristian Fiskerstrand
> http://www.sumptuouscapital.com
> Twitter: @krifisk
> ----------------------------
> Corruptissima re publica plurimæ leges
> The greater the degeneration of the republic, the more of its laws
> ----------------------------
> This email was digitally signed using the OpenPGP
> standard. If you want to read more about this
> The book: Sending Emails - The Safe Way: An 
> introduction to OpenPGP security is now 
> available in both Amazon Kindle and Paperback 
> format at
> http://www.amazon.com/dp/B006RSG1S4/
> ----------------------------
> Public PGP key 0xE3EDFAE3 at http://www.sumptuouscapital.com/pgp/
> 
> 
> _______________________________________________
> Sks-devel mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/sks-devel