lvm module can't see USB disks' volume group

[Alexandre Oliva]

Hi, there!

I've recently got back to using LVM to hold more than one btrfs per
disk, after a long time using btrfs on disk partitions.  I was pretty
happy that GRUB could deal with /boot being part of even encrypted root
filesystems.  Now, I wanted to go partitionless on my servers (to save
myself from hassles when moving disks into or out of USB3 enclosures
that change logical sector sizes from 512b to 4KiB), with a single
whole-disk physical volume per disk, and I found out GRUB
(grub2-2.02-0.38.fc25.x86_64) apparently wasn't up to that yet.  Oh,
bummer.

So I figured I'd have just one big LVM physical volume per disk, in
addition to the initial EF02 GPT partition that enables grub to do its
magic.  So far so good, until I decided to make two LVs in USB disks
part of the btrfs(raid1) root, and part of /boot happened to end up in
those two disks.  

GRUB couldn't boot from that root any more, and the reason seems to be
that the lvm module can't see the logical volumes in the external disks,
even though it can see the partitions of that disk as (hd5,gpt[12]) or
(fd0,gpt[12]) depending on how I configure it in the BIOS (no Libreboot
for those servers yet).

GRUB can detect the partition sizes and report them as an unknown
filesystem type all right, but it apparently won't look into them as PVs
so as to expose the LVs in them as (lvm/<VG>-<LV>).  That's unlike the
internal SATA disks, and even an old PATA disk on a PCI controller: the
PVs in these are correctly inspected and made visible as (lvm/*) units.

I painted myself back out of that corner by booting out of an alternate
root and removing the external disks from the btrfs root, for the time
being, but I would really like to be able to use disks in USB enclosures
to hold root (containing boot), so I started looking for anything that
would detect and discard e.g. usb disks, or disks marked as removable,
or any such thing in the lvm and diskfilter modules, and I came out
empty-handed.

I even tried to force diskfilter and lvm to be loaded after biosdisk in
the embedded grub img, hoping that would move lvm detection to a time in
which all disks were already visible (assuming non-USB disks were made
visible earlier somehow), but that didn't help.

I'm puzzled as to how to proceed: I suppose I could add a non-LVM
partitition to hold /boot, or move to a separate /boot only in internal
disks, or some other such arrangements, but since I'm ultimately aiming
at partitionless LVM, and meanwhile I'd like to have as few partitions
as possible, I'd appreciate any suggestions as to how to try to get grub
to see LVs in the USB disks.  (Heck, even one of my root backups, that I
wish I could have used to paint myself out of the unbootable
btrfs(raid1) root, was in there and I couldn't get to it!  Scary! :-)

Suggestions as to how to hack grub into booting out of partitionless,
whole-disk PVs (without additional storage) with current grub would be
welcome too, of course ;-)


Note: I haven't yet determined that it's the USBness of the disks that's
a factor; it might be disk size, though I find that unlikely.  So far,
I've only converted some sub-TB disks to LVM.  The new USB disks that I
set up with LVM are 5TB with 512b logical sectors; I've got another
slightly older 5TB non-USB disk that I intend to convert over the next
few days, and other smaller but still few-TB-sized disks, some plain
SATA, some in USB3 enclosures that make the disks seem to support only
4KiB sectors, and so I may shortly know whether GRUB has issues with big
PVs or USB disks.  In case you already know, I'd appreciate any warnings
to keep me away from big trouble ;-)


Thanks, and keep up the great work,

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist|Red Hat Brasil GNU Toolchain Engineer