Re: strange grub error on ARM64

[Michael Chang]

On Mon, Mar 06, 2023 at 04:22:53PM -0800, Atish Patra wrote:
> On Mon, Mar 6, 2023 at 1:25 PM Fabian Vogt <fvogt@suse.de> wrote:
> >
> > Hi,
> >
> > Am Montag, 6. März 2023, 21:59:23 CET schrieb Daniel Kiper:
> > > Adding Gary and Michael...
> > >
> > > Gary, Michael, could you help with this? Or at least forward this email
> > > to folks who may help...
> > >
> > > Daniel
> > >
> > > On Mon, Mar 06, 2023 at 10:56:52AM -0800, Atish Patra wrote:
> > > > Hi,
> > > > I am working on a cross-architecture cleanup series that removes the
> > > > arch specific headers from RISC-V, ARM, ARM64 and unifies the linux
> > > > loader on ARM64 & Linux.
> > > >
> > > > The v7 version can be found here.
> > > > https://lists.gnu.org/archive/html/grub-devel/2023-02/msg00046.html
> > > >
> > > > It had a few bugs which are fixed and can be found here.
> > > >
> > > > Here are the patches
> > > > https://github.com/atishp04/grub/commits/riscv_uefi_v8
> > > >
> > > > Anybody with ARM64 and/or test setup: Can you please do a quick test
> > > > to see if you see similar or any other failure ?
> > > >
> > > > I noticed a very strange error during ARM64 testing. I have probably
> > > > done something stupid during the grub build
> > > > as I can reproduce this with upstream grub as well.
> > > >
> > > > I was trying to boot an OpenSuse image in qemu virt machine for ARM64 
> > > > as per
> > > > the instructions[1]. The stock grub image works fine however my custom
> > > > build fails to boot
> > > > and drops to the grub command line. It seems the grub detects the disk
> > > > but doesn't read the root partition correctly on my cross-compiled 
> > > > image.
> > > > It has one additional directory ("@") which I don't see when I mount
> > > > the disk. it doesn't list any files as well.
> >
> > What you see is the expected behaviour with upstream GRUB.
> > The btrfs volume contains various subvolumes below @ which are mounted into 
> > the
> > root file system at various points. The root filesystem itself is at
> > @/.snapshots/<number>/snapshot/ and set as the default subvolume. That's 
> > what
> > is mounted in a running system if you pass default flags. To see the same as
> > GRUB, you can do mount -o subvolume=/ /dev/foo3 /mnt.
> >
> 
> Thanks!! Thank god I am not crazy as I thought I was ;)
> 
> > Downstream there's a patch (and multiple related ones) that makes GRUB mount
> > the default subvolume instead of the root of the filesystem:
> > https://build.opensuse.org/package/view_file/openSUSE:Factory/grub2/grub2-btrfs-03-follow_default.patch?expand=1&rev=285
> >
> 
> Is this the only patch or I need to pull all the patches on that tree
> to make it work ?

You'll need other patch so that files not in default subvolume can be
accessed through mount at grub's runtime, also to enable automatic setup
via the utility grub-(install|mkconfig).

> 
> Is there an easy way to test upstream grub stuff on ARM64 in case I
> need to pull more dependent patches to make it work?

You can also use rpm from this Open Build Service (OBS) project:

https://build.opensuse.org/package/show/home:michael-chang:grub:devel/grub2

It is for merging upstream git with SUSE patches rebased regularly per
month convenient to catch early build failure, find regression, try out
new feature and so on.

Thanks,
Michael

> 
> > The design of the "Boot into Snapshot" functionality relies on this.
> >
> > > > ============= Cross compiled grub image =====================
> > > > grub> ls -l (hd0,gpt3)/
> > > > DIR          20230302080325 @/
> > > >
> > > > grub> ls -l (hd0,gpt3)/@/
> > > > DIR          20230302080326 boot/
> > > > DIR         srv/
> > > > DIR          20230302080326 usr/
> > > > DIR         opt/
> > > > DIR          20230302080326 home/
> > > > DIR         var/
> > > > DIR         tmp/
> > > > DIR         root/
> > > >
> > > > grub> ls -l (hd0,gpt3)/@/boot/
> > > > DIR          20230302080326 grub2/
> > > >
> > > > grub>
> > > > ============= Stock grub image from the OpenSuse 
> > > > image=====================
> > > > grub> ls (hd0,gpt3)/
> > > > dev/ run/ boot/ config.partids lib sbin srv/ usr/ opt/ 
> > > > config.bootoptions lib64
> > > > .snapshots/ bin home/ etc/ var/ tmp/ sys/ mnt/ proc/ root/
> > > >
> > > > The same build steps works fine for RISC-V for OpenSuse image though.
> > > > However, I couldn't figure out what it was. Any help is appreciated.
> >
> > It's possible that it does not use snapshot or not even btrfs.
> >
> > Cheers,
> > Fabian
> >
> > > > [1] https://en.opensuse.org/openSUSE:AArch64
> > > >
> > > > --
> > > > Regards,
> > > > Atish
> > >
> > > _______________________________________________
> > > Grub-devel mailing list
> > > Grub-devel@gnu.org
> > > https://lists.gnu.org/mailman/listinfo/grub-devel
> >
> >
> 
> 
> -- 
> Regards,
> Atish