bug#17676: btrfs subvolumes and bind-mounts make df report incorrect and/or extra results

[Pádraig Brady]

On 06/03/2014 04:21 PM, David Schleimer wrote:
> When you have btrfs mount where there is a both a subvolume of that mount, 
> and the source of a bind-mount under that mount, df will report confusing 
> results.  It will show results for the bind-mount instead of the main btrfs 
> mount when you pass the subvoulme on the command line.  It may (depending on 
> the system) show both the bind-mount and main mount in bare df results.
> 
> I've attached a script with the minimal repro instructions.  It needs to be 
> run as root since it (temporarily) mounts filesystems under your tmpdir.
> 
> I tested on two machines:
> Fedora 20 running kernel 3.14.4-200.fc20.x86_64, against both the system df 
> 8.21 and df 8.22 built from source.
> 
> Centos 6.4 running a 3.10.39 variant, with significant backports, notably 
> including most btrfs changes from mainline.  Testing against the system df 
> 8.4, and df 8.22 again built from source.
> 
> On the fedora machine, /etc/mtab is a symlink to /proc/self/mounts.  We see 
> output that refers to the correct block device in all cases.  However, we see 
> a report for the bind-mount destination instead of the main btrfs mount when 
> asking specifically for the subvolume path, and see results for both the main 
> btrfs filesystem and the bind-mount when we run a bare df.
> 
> Transcript of Fedora output:
> 
> address@hidden dschleimer]# ./repro_instructions.sh
> 1024+0 records in
> 1024+0 records out
> 10485760 bytes (10 MB) copied, 0.00963354 s, 1.1 GB/s SMALL VOLUME: forcing 
> mixed metadata/data groups
> 
> WARNING! - Btrfs v3.12 IS EXPERIMENTAL
> WARNING! - see http://btrfs.wiki.kernel.org before using
> 
> Turning ON incompat feature 'mixed-bg': mixed data and metadata block groups 
> Turning ON incompat feature 'extref': increased hardlink limit per file to 
> 65536 Created a data/metadata chunk of size 1048576 fs created label (null) 
> on block
>         nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.12 
> Create subvolume 'filesystem/subvolume'
> 
> 
> 
> Begin unexpected output
> Filesystem     1K-blocks  Used Available Use% Mounted on
> /dev/loop0         10240    36      6112   1% /tmp/tmp.biUryZcwD9/binddest
> Expected report for the main btrfs mount, not the bind-mount
> /dev/loop0              btrfs        10240       36      6112   1% 
> /tmp/tmp.biUryZcwD9/filesystem
> /dev/loop0              btrfs        10240       36      6112   1% 
> /tmp/tmp.biUryZcwD9/binddest
> Expected only one btrfs filesystem
> /home/dschleimer
> address@hidden dschleimer]# 
> PATH=/home/dschleimer/Downloads/coreutils-8.22/src/:$PATH 
> ./repro_instructions.sh
> 1024+0 records in
> 1024+0 records out
> 10485760 bytes (10 MB) copied, 0.012752 s, 822 MB/s SMALL VOLUME: forcing 
> mixed metadata/data groups
> 
> WARNING! - Btrfs v3.12 IS EXPERIMENTAL
> WARNING! - see http://btrfs.wiki.kernel.org before using
> 
> Turning ON incompat feature 'mixed-bg': mixed data and metadata block groups 
> Turning ON incompat feature 'extref': increased hardlink limit per file to 
> 65536 Created a data/metadata chunk of size 1048576 fs created label (null) 
> on block
>         nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MiB Btrfs v3.12 
> Create subvolume 'filesystem/subvolume'
> 
> 
> 
> Begin unexpected output
> Filesystem     1K-blocks  Used Available Use% Mounted on
> /dev/loop0         10240    36      6112   1% /tmp/tmp.TEBR2D9Vta/binddest
> Expected report for the main btrfs mount, not the bind-mount
> /dev/loop0              btrfs        10240       36      6112   1% 
> /tmp/tmp.TEBR2D9Vta/filesystem
> /dev/loop0              btrfs        10240       36      6112   1% 
> /tmp/tmp.TEBR2D9Vta/binddest
> Expected only one btrfs filesystem
> /home/dschleimer
> 
> On Centos, /etc/mtab is a plain file.  When asking specifically for the 
> subvolume, we see a report for the bind-mount dest which lists the bind-mount 
> source as the filesystem.  When we run a bare df, we see only one record with 
> the file backing the loopback device as the filesystem, but the bind-mount as 
> the mount-point.
> 
> Centos transcript:
> 
> [08:03:09 Tue Jun 03 2014] address@hidden /home/dschleimer dschleimer  282 $ 
> ./repro_instructions.sh
> 1024+0 records in
> 1024+0 records out
> 10485760 bytes (10 MB) copied, 0.0101568 s, 1.0 GB/s
> 
> WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see 
> http://btrfs.wiki.kernel.org before using
> 
> SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata 
> chunk of size 1048576 fs created label (null) on block
>         nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs 
> v0.20-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume'
> 
> 
> 
> Begin unexpected output
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /tmp/tmp.V4FNWcBXke/filesystem/subvolume/bindsource
>                          10240        36      6112   1% 
> /tmp/tmp.V4FNWcBXke/binddest
> Expected report for the main btrfs mount, not the bind-mount
> df: `/mnt/gvfs': Function not implemented /tmp/tmp.V4FNWcBXke/block
>              btrfs       10240        36      6112   1% 
> /tmp/tmp.V4FNWcBXke/filesystem
> Expected only one btrfs filesystem
> /home/dschleimer
> 
> [08:09:39 Tue Jun 03 2014] address@hidden /home/dschleimer dschleimer  282 $ 
> PATH=/data/users/dschleimer/coreutils-8.22/src/:$PATH ./repro_instructions.sh
> 1024+0 records in
> 1024+0 records out
> 10485760 bytes (10 MB) copied, 0.0116229 s, 902 MB/s
> 
> WARNING! - Btrfs v0.20-rc1-358-g194aa4a IS EXPERIMENTAL WARNING! - see 
> http://btrfs.wiki.kernel.org before using
> 
> SMALL VOLUME: forcing mixed metadata/data groups Created a data/metadata 
> chunk of size 1048576 fs created label (null) on block
>         nodesize 4096 leafsize 4096 sectorsize 4096 size 10.00MB Btrfs 
> v0.20-rc1-358-g194aa4a Create subvolume 'filesystem/subvolume'
> 
> 
> 
> Begin unexpected output
> Filesystem                                          1K-blocks  Used Available 
> Use% Mounted on
> /tmp/tmp.o7w6NcN178/filesystem/subvolume/bindsource     10240    36      6112 
>   1% /tmp/tmp.o7w6NcN178/binddest
> Expected report for the main btrfs mount, not the bind-mount
> /tmp/tmp.o7w6NcN178/block                        btrfs      10240         36  
>      6112   1% /tmp/tmp.o7w6NcN178/filesystem
> Expected only one btrfs filesystem
> /home/dschleimer


So the crux of the confusion is that the subvolume has a different device ID.
I.E. a separate virtual file system within the main file system,
that shares storage but are otherwise treated like separate file systems
for hard links etc. Here are the device IDs for your test setup:

  # find -printf "%D %p\n"
  34 .
  63 ./binddest
  62 ./filesystem
  63 ./filesystem/subvolume
  63 ./filesystem/subvolume/directory
  63 ./filesystem/subvolume/bindsource
  34 ./block

When you expose that virtual file system to df
with a bind mount it will be identified separately as you've shown.
BTW it's not specific to bind mounts, as you can see the same
thing by mounting the subvolume separately like:
  mkdir submntdest
  mount -o subvolid=256 -o loop block submntdest
It could be argued that since there are explicit mounts to this
subvolume that one (the shortest) of these is the most appropriate to display?

If you don't mount separately, then df notices that ./filesystem/subvolume is
a "separate mount" due to the different device id, and will display that
as the mount dir and "-" as the device which isn't ideal either.

It's more problematic that the btrfs file system is listed
multiple times, especially with --total which will be incorrect.
We might be able to avoid this problem by taking the first
entry for duplicate device names, when those device names exist.
However I'm worried that there are cases where separate file systems
with separate storage (distinguished by mount options) have the
same device listed. Maybe we could suppress these duplicates by
scanning /proc/self/mountinfo and discarding entries that didn't
have '/' in the fourth column, though that's awkward and I'm
not sure how general it is either.

thanks,
Pádraig.