Re: PPC64

[Hollis Blanchard]

Please CC me, since I'm no longer subscribed to grub-devel.

> From: Manoel <address@hidden>
> To: The development of GRUB 2 <address@hidden>
> Subject: Re: PPC64
> Date: Tue, 21 Oct 2008 14:43:25 -0200
> 
> Hi Hollis,
> 
> On Mon, 2008-10-20 at 14:32 -0500, Hollis Blanchard wrote: 
> > On Mon, 2008-10-20 at 17:18 -0200, Manoel wrote:
> > > 
> > > I'm working in a project to use grub2 to boot some ppc machines(p6 , p5
> > > and so on) and we had some difficulties with a grub modules problem.
> > > Grub fails to load modules.
> > > 
> > > When debugging I noted that grub try to find the headerinfo modules
> > > struc (which is identified by the magic number 0x676d696d) in the
> > > address 0x2d000 (_end + gap aligned in 4k blocks).
> > > but this address does not contains the headerinfo.
> > > 
> > > So i altered the source code such as the memory is searched to find the
> > > magic number. It is then found at address 0x38f4c and then grub find
> > > some modules (but fails after) has showed in attachment grub2.txt.
> > 
> > ...
> > ../kern/dl.c:527: module at 0x3e0dc, size 0xc9c
> > ../kern/dl.c:556: relocating to 0x28720
> > ../kern/dl.c:513: flushing 0x4 bytes at 0x28190
> > ../kern/dl.c:513: flushing 0x14 bytes at 0x281d0
> > ../kern/dl.c:513: flushing 0x68 bytes at 0x28220
> > ../kern/dl.c:513: flushing 0x410 bytes at 0x282c0
> > ../kern/dl.c:570: module name: amiga
> > ../kern/dl.c:571: init function: 0x282c0
> > ../kern/dl.c:527: module at 0x3ed84, size 0xe28
> > ../kern/dl.c:556: relocating to 0x280a0
> > ../kern/dl.c:513: flushing 0x4 bytes at 0x27a30
> > ../kern/dl.c:513: flushing 0x14 bytes at 0x27a70
> > ../kern/dl.c:513: flushing 0xfc bytes at 0x27ac0
> > ../kern/dl.c:513: flushing 0x458 bytes at 0x27bf0
> > ../kern/dl.c:570: module name: apple
> > ../kern/dl.c:571: init function: 0x27bf0
> > ../kern/dl.c:527: module at 0x3fbb8, size 0xeca4
> > ../kern/dl.c:556: relocating to 0x27940
> > 
> > Notice how much larger that last module is than the ones before it.
> > That's a bit suspicious... do you have any modules that size?
> > 
> 
> I'd like to address this issue later but their size are really messed
> up. Grub can find the modules (how you can see by the modules names)
> though. The modules should have 7k at most but grub identified them has
> having about 50k.

That is really strange. I wonder if you have an ABI issue like
sizeof(long)... how is the grub-mkimage tool compiled? Please make sure
it's 32-bit. The grub binary that executes at boot should also 32-bit.

> I'm also curious why we must have a GAP between _end and the modules.
> Why do not put the modules right after the _end address.

We put the modules into a separate PT_LOAD ELF segment, and these must
be aligned.

One other possibility is that your firmware doesn't like the way
grub-mkimage throws out the section table on the ELF file. You could try
changing that behavior.

I suppose you could also try to extend the existing PT_LOAD segment
instead of creating a new one, but architecturally creating a new
segment for the modules is much nicer.

> I need to look more into the source code but I noted the modules are
> allocated in address in a decrementing order. The next module is always
> loaded in a address below the previous module. I don't know if this
> memory is allocated by the OF or if Grub forces the address to load the
> modules this way.
> How I have said before that I will look at this issue after the modules
> header info location address issue is resolved.
>  
> > > that address calculation led me to believe that I can tell where the
> > > struct will be on memory based in its place in the binary.
> > > 
> > > I also noted that basemod ( indicates where the modules sections begin)
> > > used by grub_mkelfimage is the same calculated by grub (_end + GAP). but
> > > it seems to not store it on the necessary address.
> > > 
> > > using hexedit I could see that in the address 0xCC98 (in the file
> > > generated by grub_mkelfimage) is stored the struct header info.
> > 
> > Well, hmm. Given the readelf output below, file offset 0xcc98 should be
> > loaded right at 0x2d000. Since you can see the magic number there
> > (correct?), I can't explain why the ELF loaded places it at 0x38f4c.
> 
> Yes, the magic number is exactly at the address 0xcc98 on the file
> generated by the grub_mkelfimage. How can you tell the address it should
> appear in memory based on its address in file? Maybe it's only valid in
> some old OF version? 

Look at the segment list again:
> Program Headers:
>  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>  LOAD           0x0000b4 0x00010000 0x00010000 0x0cbe4 0x14098 RWE 0x10
>  GNU_STACK      0x00cc98 0x00000000 0x00000000 0x00000 0x00000 RWE 0x4
>  LOAD           0x00cc98 0x0002d000 0x0002d000 0x5c1c8 0x5c1c8 RWE 0x4
>  NOTE           0x068e60 0x00000000 0x00000000 0x0002c 0x00000 R   0x4

Offset tells you where in the file each segment begins. FileSiz is how
many bytes to read from the file. VirtAddr/PhysAddr is where in memory
to copy it, and MemSiz is how many bytes the segment will occupy in
memory. (If MemSiz > FileSiz, the trailing bytes are zero.)

0x38f4c, where you found the header, is about 50KB inside the second
LOAD segment (which was added by grub-mkimage and contains the modules).
Either the firmware's ELF loader did something bad loading the file, or
grub-mkimage did something bad constructing the file.

Since you said you can manually verify that file offset 0xcc98 does in
fact contain the magic number, and we can all see that it *should* be
loaded at 0x2d000, that makes it seem like the loader did something
wrong.

Can you report the bug to the firmware team, supply the broken binary,
and see if they'll take a look at it?

By the way, what filesystem is GRUB located on, and how big is the
partition? Historically IBM firmware has had bugs loading from many
filesystems, but I think FAT12 is OK as long as it's on a small
partition.

> > Can you report what memory firmware is using on this system? IIRC you
> > can decode it from the "available" properties in the memory nodes.
> > 
> I couldn't find any apparently useful information in the memory nodes
> properties. I have attached it anyway, I have also attached the "/" node
> properties. 

I didn't get these.

You'll need to refer to the PowerPC and CHRP bindings to IEEE 1275 to
interpret the /memory/available properties. I don't remember the field
widths off the top of my head, but they are basically a list of <base,
length> pairs that denote regions of memory available to applications.

I was just wondering if maybe firmware was occupying the memory
specified in the ELF header for the modules segment, and was doing
something stupid after that.

-- 
Hollis Blanchard
IBM Linux Technology Center