Re: [PATCH] disk/mdraid1x: Fix >2TB RAID detection with BIOS

[Robert LeBlanc]

On Fri, Nov 18, 2016 at 10:31 AM, Andrei Borzenkov <address@hidden> wrote:
> 18.11.2016 19:49, Robert LeBlanc пишет:
>> Based on debug info, GRUB is requesting to read the end of the drive
>> to read the mdadm 1.0 superblock and gets the following message:
>>
>> kern/disk.c:421: Read out of range: sector 0x27fffffc8 (attempt to
>> read or write outside of disk 'hd0').
>>
>
> Is MD RAID on whole disk or partition? What partition label - MBR or
> GPT? Could you post fdisk -l from Linux (if your fdisk does not support
> GPT, use "gdisk -l /dev/sdX")?

# gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
 MBR: protective
 BSD: not present
 APM: not present
 GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 10737418240 sectors, 5.0 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 939F7AAD-AADD-481D-8896-DA0B4033C70C
Partition table holds up to 128 entries
First usable sector is 2048, last usable sector is 10737418206
Partitions will be aligned on 2048-sector boundaries
Total free space is 0 sectors (0 bytes)

Number  Start (sector)    End (sector)  Size       Code  Name
  1            2048            4095   1024.0 KiB  EF02
  2            4096     10737418206   5.0 TiB     FD00

partition # is a mdadm RAID 1 member partitioned as follows...

# sgdisk -l /dev/md123
# parted /dev/md123 print free
Model: Linux Software RAID Array (md)
Disk /dev/md123: 5497GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system     Name  Flags
       17.4kB  1049kB  1031kB  Free Space
1      1049kB  2097kB  1049kB                        bios_grub
2      2097kB  5371MB  5369MB  linux-swap(v1)
3      5371MB  6371MB  1000MB  ext2
4      6371MB  5497GB  5491GB  ext4
       5497GB  5497GB  966kB   Free Space

/dev/sdb is partitioned identically to /dev/sda

>
>> My understanding is that BIOS can only access 2TB of disk so this
>> makes sense. This patch turns this error from fatal in detecting mdadm
>> RAID arrays into a non-critical error so that other versions of
>> metadata can still be checked. Where I was running into the problem
>> was that we have a 5TB disk with mdadm metadata version 1.2 and we
>> could not get the md/0 to show up in GRUB because after the error, it
>> never tried to check the other mdadm metadata versions. Once this
>> patch was in place, the md/0 showed up and we were able to boot the
>> box because /boot on the mdadm RAID was below the 2TB BIOS limit.
>>
>> mdadm metadata 1.0 superblock is located 8-12KB from the end of the disk
>> mdadm metadata 1.1 superblock is located at sector 0 of the disk/partition
>> mdadm metadata 1.2 superblock is located at sector 8 of the disk/partition
>>
>> The loop was
>> for i=0; i<3; i++{
>>   check for the metadata
>>   if error {
>>     return NULL
>>   }
>>   return superblock info
>> }
>>
>> So if an error occurs when checking one type of metadata, it does not
>> try the other types to see if they exist. This patch fixes that.
>>
>> I hope that clears things up, let me know if you have additional questions.
>> ----------------
>> Robert LeBlanc
>> PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1
>>
>>
>> On Thu, Nov 17, 2016 at 9:16 PM, Andrei Borzenkov <address@hidden> wrote:
>>> 17.11.2016 23:09, Robert LeBlanc пишет:
>>>> When a mdadm RAID array is on a drive larger than 2TB, the array is not
>>>> able to be detected and as such even if the array has a partition that
>>>> holds /boot under the 2TB limit, it is unable to boot the machine. This
>>>> is caused by metadata 1.0 being tested first which allocates the
>>>> superblock at the end of the device. When it tries to access the end of
>>>> the device it throws an error and the code returns without trying to
>>>
>>> Why read returns error here and how it is related to device size?
>>>
>>>> find the superblock at other locations (metadata 1.1 and 1.2). This
>>>> patch changes the error to not be fatal and allow for the checking for
>>>> the other metadata versions and allowing the machine to boot as long as
>>>> /boot is under the 2TB BIOS limit. This won't cause issues with 1.0
>>>> metadata because GRUB is able to read the partitions from the front of
>>>> the drive/partition without having to determine the data offset, since
>>>> the data for metadata 1.0 starts at sector 0.
>>>>
>>>> Signed-off-by: Robert LeBlanc <address@hidden>
>>>> ---
>>>>  grub-core/disk/mdraid1x_linux.c | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/grub-core/disk/mdraid1x_linux.c 
>>>> b/grub-core/disk/mdraid1x_linux.c
>>>> index 7cc80d3..cc7350c 100644
>>>> --- a/grub-core/disk/mdraid1x_linux.c
>>>> +++ b/grub-core/disk/mdraid1x_linux.c
>>>> @@ -148,7 +148,7 @@ grub_mdraid_detect (grub_disk_t disk,
>>>>
>>>>        if (grub_disk_read (disk, sector, 0, sizeof (struct 
>>>> grub_raid_super_1x),
>>>>                         &sb))
>>>> -     return NULL;
>>>> +     continue;
>>>>
>>>>        if (sb.magic != grub_cpu_to_le32_compile_time (SB_MAGIC)
>>>>         || grub_le_to_cpu64 (sb.super_offset) != sector)
>>>>
>>>
>

----------------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1