bug-fileutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Problems with DU and DF


From: Bob Proulx
Subject: Re: Problems with DU and DF
Date: Wed, 7 Feb 2001 18:53:34 -0700

Randy

> While df will show the disks getting full, I use du to determine
> which files need truncating. Once located I will either truncate
> them or just remove them via the rm command. However, after removing
> 150 - 200 megs of log files a df command does not show the correct
> free space on the filesystem.

I believe the issue you are seeing is that the filesystem uses a
reference counting scheme to free up disk space.  Essentially it is
garbage collection applied to disk blocks instead of memory pages.
When reference count goes to zero the disk blocks are freed.

> If, for example, I have a filesystem mounted as /var with log files being 
> written on it and a df shows 95% used and I delete 200 megs of files the df 
> will still show 95% full for many days. Is there anyway to force it to 
> 'reset' or 'reread' the filesystem and get the correct numbers? If I add a 
> file, no matter what size, a df will show it immediately.. the problem is 
> only with removing them. Or am I just removing them incorrectly?

You mention log files.  That triggered my remark about reference
counting.  If you have a program which writes a log file and keeps
that file open for writing then removing the file will not free up the
disk space.  The filesystem still sees the program as having a
reference to it.  Therefore the filesystem will not free up that disk
space.

Actually removing the file is the wost possible thing in that case
since afterward there is no way to do anything with that disk space.
Except to kill and restart the program holding the file open.  Killing
the program will close the file and drop the reference count by one.
When the reference count goes to zero then the filesystem will free
the file and it it will no longer be reported with df.

The df command reports the filesystem disk free space.  The du command
reports the disk used under a directory.  As you can see if a file is
open and the reference count is non-zero but it is not contained in
any directory then these two numbers are not the same numbers.  Also
note that df usually reports relative to the filesystem minfree
parameter which is usually 5-10% of the total disk space.

Instead of removing log files I have always truncated them to avoid
the exact problem you are describing.  One way to truncate them is
with shell output redirection.

  : > logile

[I am pedantic and so I use ":" (aka true) to be a command to
redirect.  You might have seen this in the past without any command
such as "> logfile".  That works everywhere I know of but I believe
the POSIX shell standard only requires redirections to redirect the
I/O of commands.  Therefore I always include a command to redirect and
the simplest command is ":".]

The net result is that the disk space is immediately freed and it does
not matter if other programs have references to the file.

> Running du -s /var  always seems to show what I would consider the correct 
> space used and changes proportionately with the sizes of the files removed.

That seems strange and contradictive.  I believe the following should
generally print the same number.  This is psuedo-code and not real
script code.  But you should be able to understand what I mean.

  `ls -ld /var`/1024 + `du -sk /var`
    == `du -sk /var/* | awk '{total+=$1}END{print total}'`

Bob



reply via email to

[Prev in Thread] Current Thread [Next in Thread]