Re: [Findutils-patches] [PATCH] find memory leak

[Goffredo Baroncelli]

On 2017-02-07 02:04, Bernhard Voelker wrote:
[...]
> 
> Okay, so back to the question why free_file_system_list() is invoked that
> often: usually, filesystem_type() would cache the result ... 

I found the problem. I using btrfs as my filesystem, which has an inconsistency 
between /proc/self/mountinfo and stat.

$ cat /proc/self/mountinfo  | grep btrfs
20 0 0:18 /debian / rw,noatime,nodiratime shared:1 - btrfs /dev/sdc3 
rw,ssd,space_cache,subvolid=257,subvol=/debian
73 20 0:18 /boot /boot rw,noatime,nodiratime shared:29 - btrfs /dev/sdc3 
rw,ssd,space_cache,subvolid=299,subvol=/boot
75 20 0:18 / /var/btrfs rw,noatime,nodiratime shared:30 - btrfs /dev/sdc3 
rw,ssd,space_cache,subvolid=5,subvol=/
$ stat /
  File: /
  Size: 280             Blocks: 0          IO Block: 4096   directory
Device: 14h/20d Inode: 256         Links: 1
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2015-09-06 12:06:36.151128298 +0200
Modify: 2017-01-15 22:44:46.869717588 +0100
Change: 2017-01-15 22:44:46.869717588 +0100
 Birth: -

/proc/self/mountinfo for "/" has 0:18, but stat shows 0:20

$ stat /boot/
  File: /boot/
  Size: 1202            Blocks: 0          IO Block: 4096   directory
Device: 2bh/43d Inode: 256         Links: 1
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2015-08-29 13:21:52.035986005 +0200
Modify: 2017-02-05 00:18:07.351236324 +0100
Change: 2017-02-05 00:18:07.351236324 +0100
 Birth: -

/proc/self/mountinfo for "/boot" has 0:18, but stat shows 0:43


(See for further detail

https://www.spinics.net/lists/linux-btrfs/msg58908.html)

So apart the leak, "find" under btrfs has bad performance due to the 
inconsistency between /proc/self/mountinfo and stat()..


> and I'm assuming
> that all of those 16269 files are on the same mounted file system.
> A look into the underlying file_system_type_uncached() gives the answer:
> either stat() of the mount point of /lib/modules fails, or the returned
> 'st_dev' number does not match the entry in the mount list - see the
> following lines.
> 
> What do you get in the /tmp/find.strace.log file when you run the command
> via strace:
> 
>   $ strace -e stat -vo /tmp/find.strace.log ./find ...
> 
> ?
> 
> I'm getting 7x lines like this:
> 
>   stat("/", {st_dev=makedev(8, 2), ...
> 
> and then searching for the above major:minor (8, 2) device in the mountinfo
> file unveils 'ext4' here (which is not pruned):
> 
>   $ grep ' 8:2 ' /proc/self/mountinfo
>   35 0 8:2 / / rw,noatime shared:1 - ext4 /dev/sda2 rw,data=ordered
> 
> Somewhere in the above chain, find fails to determine the file system type
> on your system and therefore re-tries for each entry.  Without the memleak
> patch, it's clear that the process would grow excessively.
> Can you confirm?
> 
> Thanks & have a nice day,
> Berny
> 
> 


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5