Re: Problem with ISO9660 and files stored on multiple extents

[Andrei Borzenkov]

16.02.2017 20:42, Carlo Caione пишет:
> On Thu, Feb 16, 2017 at 5:52 PM, Andrei Borzenkov <address@hidden> wrote:
>> 16.02.2017 19:04, Vladimir 'phcoder' Serbinenko пишет:
>>> On Thu, Feb 16, 2017, 16:56 Carlo Caione <address@hidden> wrote:
> 
>>> Then it's possible that your patch is correct. I don't know why the format
>>> changes or what the 0 means in this field
>>>
>>
>> Looking at current squash tools git, 0 is valid fragment number. Any
>> chance we hit different file type here? Carlo, could you make your
>> squashfs image available?
> 
> Here the whole non-booting ISO
> https://d1anzknqnc1kmb.cloudfront.net/eos-amd64-amd64/eos3.1/sea/170214-155339/eos-eos3.1-amd64-amd64.170214-155339.sea.iso
> (`endless/endless.squash` is the squash image)
> `qemu-system-x86_64 -m 512 -machine q35,accel=kvm -hda $iso` drops you
> into a GRUB shell (same booting with EFI). For smaller images it boots
> fine.
> 
> We actually solved this problem creating the squashfs passing the
> option `-no-fragments` to `mksquashfs`. So I guess that really the
> squash4 module has some bug when dealing with big images with a number
> of fragments !=0.
> 

Not quite. We assume that fragment and block list are mutually
exclusive, while squashfs can pack file tail that does not fill full
block as fragment. So we actually need to do direct_read() followed by
the rest of grub_squash_read_data() if fragment is not invalid. Your
patch worked by accident (because there was only one fragment with
number 0) and actually made it truncate file.

I need to wrap my head around our code unless Vladimir beats me (I'm
sure he will :) )

For reference, here is what unsquashfs does

        for(i = 0; i < inode->blocks; i++) {
... read blocks etc. This is our direct_read
        }

        if(inode->frag_bytes) {
...
                s_ops.read_fragment(inode->fragment, &start, &size);
                block->buffer = cache_get(fragment_cache, start, size);
                block->offset = inode->offset;
                block->size = inode->frag_bytes;
                queue_put(to_writer, block);
        }

inode->frag_bytes != 0 if fragment != 0xffffffff and file size is not
exact multiple of blocks.

                        i.frag_bytes = inode->fragment ==
SQUASHFS_INVALID_FRAG
                                ?  0 : inode->file_size % sBlk.s.block_size;