Re: [PATCH][UPDATED] support for xz compression format

[address@hidden]

On Tue, Feb 16, 2010 at 3:28 PM, Seth Goldberg <address@hidden> wrote:
> y
>
> Quoting address@hidden, who wrote the following on Tue, 16 Feb 2010:
>
>> On Tue, Feb 16, 2010 at 12:58 PM, Szymon Janc <address@hidden> wrote:
>>>
>>> Dnia wtorek 16 luty 2010 o 14:44:45 Vladimir 'φ-coder/phcoder' Serbinenko
>>> napisał(a):
>>>
>>>> + #define VLI_MAX_DIGITS 9
>>>> Are you sure it's 9? It gives only 63 and not 64 bits
>>>
>>> It is a limitation of xz format.
>>>
>>>> +   if (! test_header(file) || ! test_footer(file))
>>>> +   {
>>>> Seeking to the end of file is very expensive on pxe. Can it be skipped?
>>>
>>> xz stores uncompressed data size at the end of file, without it file size
>>> can't
>>> be set.
>>>
>>> gzio has same issue, it reads last 4 bytes of file to determine
>>> uncompressed
>>> data size.
>>
>> Then maybe use of a more sensible format should be strongly
>> encouraged.  For sequentially accessed devices like tape, it may be
>> expensive to return to the header to add the size.  For flash memory,
>> it may require an overwrite (and flash memory has fast random access,
>> this is pretty much the only case to prefer a footer).  And formats
>> which support pipelining of course can't rewind to update a header.
>> But in general the archive will be read many times for each time it is
>> created, so it's best to put the size in the place which is best for
>> the decompressor.
>>
>> Since gzip format allows decompression in pipeline mode, it's a
>> virtual certainty that nothing from the footer is required for
>> processing.
>
>  I know I've said this before, but it bears repeating: Don't forget about
> network booting!  This is precisely the problem for which we came up with a
> different solution for loading large compressed archives via tftp when PXE
> booting.  The assumption, of course, is that you can size a buffer maximally
> (with some sane maximum)  then shrink it once the archive has been
> decompressed.  That's what we do and it's served us will with Legacy GRUB
> for years now.

In the case of .xz, there appear to be block headers, where the
uncompressed size is stored up front.  So as long as separate blocks
can go into independent memory allocations, there'd be no need to
oversize the buffer.

>
>  --S
>
> _______________________________________________
> Grub-devel mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/grub-devel
>
>