Why might GRUB hang after Welcome message?

[Cedric Knight]

Hello

I've encountered a problem on a server where /boot is on a RAID
partition on sda and sdb, grub is installed to sda and sdb, but
upgrading sdc causes a hang after:

GRUB loading.
Welcome to GRUB!

and yet leaving the drive bay for sdc empty, it boots as normal.

I'm most interested in the general question: *how* can the GRUB core be
loaded and run by the MBR without it either showing the menu or a prompt?

My understanding is that the core will try to find the prefix and read
within that grub.cfg and if any stage fails it will show a grub rescue>
or grub> prompt.

So the only possibilities I can think of are:
a) the core that's being loaded is corrupt (and presumably not from
where it's recently been installed)
b) there's a mismatch between boot.img and a core (one of which is
outdated), perhaps setting wrong environment variables or disk geometry
c) some disk function, perhaps in the BIOS, is hanging (in this case the
sdc light is lit)

Full details of my case (plus a debug method I intend to use on the
server and some minor bugs) are at:
https://serverfault.com/questions/869559/grub-hangs-before-menu-after-a-raid-upgrade-how-to-debug

Some other cases under different conditions but with failure at the same
point:
*
https://forums.freenas.org/index.php?threads/stuck-at-welcome-to-grub.54860/
*
https://forum.openmediavault.org/index.php/Thread/7389-Installed-latest-ISO-now-stuck-at-Welcome-to-GRUB/
*
https://serverfault.com/questions/241109/linux-software-raid1-how-to-boot-after-physically-removing-dev-sda-lvm-md
* https://forum.manjaro.org/t/usb-boot-grub-freezes/21350
* https://bugzilla.redhat.com/show_bug.cgi?id=805747

and a couple of unanswered questions on this list:
* http://lists.gnu.org/archive/html/help-grub/2013-07/msg00054.html
* http://lists.gnu.org/archive/html/help-grub/2011-03/msg00034.html

These tend to be a few years old, so I'm wondering if upgrading from
1.99 to 2.02 will fix a bug.  I could read hundreds of source files, but
would prefer a few words of explanation...

Thanks.

CK