Re: History file clobbered by multiple simultaneous exits

[Linda Walsh]

Geoff Kuenning wrote:
> >         Instead of "junk", secure file systems mark it as needing to be
> > zeroed.  Perhaps instead of zeroing it ext3 simply marks it of zero
> > length?  Imagine, embedded in the junk are credit cards and passwords
> > and you'll begin to understand why zero pages are kept "in-stock" (in
> > memory) in the kernel so they can rapidly issue a fresh page.
>
> Perhaps I should mention that file systems are my field.
>
> While you're right that secure-deletion file systems clobber deleted
> data, that's the exception rather than the rule.  
---
        I noticed the 'extX' series picking up more features of
the xfs file system as time went along, and sorta imagined that other
file systems would pick up some of the security ones so they'd qualify
for use in a SELinux or Flask type environment.

        XFS doesn't zero the disk space when you free it -- it marks
as needing to be zero instead of just 'free'. I have the impression that
it might zero partial blocks if the fs only wrote a partial block but had
the fs extended by seeking beyond the point written.  I don't think
it was only demand-zeroed, since there were situations when a crash
could occur and the pages wouldn't have been zeroed but were marked as
part of the file's data.

> The difference from zero pages in memory is that the API for memory is
> designed around the "give me a block and I'll do things with it later"
> model.  So asking for a page doesn't inherently clobber it, and the
> kernel has to do that for you.
---
        Yeah, I was remembering (had to relook it up) about the recent
addition of huge-zero-page support and how it would save memory by allowing
many procs to share the page -- and then -- as you said earlier do the
COW, and wasn't sure of the exact impact other than the detail that
many processes needed zero'd pages as part of the address space -- I
glossed over the detail that they all share the same page... (too much
reading and spread myself a bit thin at times -- especially as the field
widens...ARG!)

  By contrast, the only way to allocate a
> page in a file system is to either write to it or to lseek over it.  The
> first case automatically clobbers it; in the second it gets marked as
> "demand zero if anybody ever reads it" but the disk never gets
> written--in fact, no block is allocated unless you write to it.
----
        You mean in ext3?  xfs tends to lay out blocks that you haven't
even asked for in various situations in order to reduce fragmentation.  In
some of those situations, it will reclaim the unused space, but it tries
to spread data out so it won't get real fragmented -- which, ironically,
is why the main fragmentation issue with xfs is with it's free space! ;-)



> (As a historical note, Unix at least through V7 didn't demand-zero newly
> allocated memory.  Eventually somebody noticed the bug and a wrote
> program to steal passwords that way, prompting a change in the kernel
> code.
====
        I'm sure.. historically nothing was done with anything..
it's all been a learning curve.


> > It's an edge case usually only seen during some system crash as you mentioned,
> > so I can't see how it would cause the symptom you are seeing, but it's only
> > seen in crashes in more mature file systems.  Can you reproduce it on another
> > file system like xfs?
>
> Good question.  Unfortunately that's not a practical experiment for me
> to run.  :-(
---
rats

I'd give the highest probability of a bug causing this being in the
fs-driver more than any place else.... oh well..

The problem w/bugs like these is that they are like looking at the behavior
of atomic particles -- the act of looking at them changes their behavior!

Best to work around the issue's than spend an unknown amount of time
(unless you really have nothing better to do)... we can't stop and debug
every problem along the path of whatever we're doing...*sigh*...and trying
to get maintainers to realize that they need to do their part to keep
their code clean and not have them dismiss you with 'patches gladly accepted',
is often a frustrating exercise in the open source community.