[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-xorriso] bug with handling symbolic links?
From: |
Joerg Meyer |
Subject: |
Re: [Bug-xorriso] bug with handling symbolic links? |
Date: |
Fri, 27 Feb 2015 11:57:14 +0100 |
Hi Thomas,
> Do you have lots of hard links in your snapshots ?
> (It's only about inodes shared inside the directory
> trees which you map into xorriso. It does not matter
> whether there are other links pointing to those
> inodes from outside those directories.)
Good question - maybe I have misunderstood and/or misinterpreted something here
as well:
Each snapshot contains on the order of 10E6 files (dominated by gentoo's
portage tree)
Less than ~10% have changed between the two snapshots included in the image
for which the memory requirements became particularly noticeable.
Identical files have different dev_t but the same ino_t (physically stored in
the same location) -
which lead me to conclude that they are subject to hardlinks processing with
-hardlinks on.
Is that correct?
> Does it really have that much impact ?
> Can you tell me how much memory both runs consume ?
Well, without having recorded accurate profiles, I can only tell you what I
remember from "manual monitoring":
For the image generation described above, -for_backup consumes >250MB of swap
(in addition to the 512MB RAM) quite early (i.e. during first -add),
whereas -acl on -xattr on -md5 only swaps later (i.e. during the subsequent
-update_r) and much more moderately (~30 MB),
The linux installation is rather slim and only consumes <50MB of RAM (after a
fresh start).
I might have a closer look when testing your latest rockridge patches...
Best wishes,
Jörg.
On 23 Feb 2015, at 19:24, Thomas Schmitt <address@hidden> wrote:
> Hi,
>
>> -hardlinks "on" included in -for_backup was what I did not want -
>> after I understood that this increases memory requirements quite noticeably.
>
> Does it really have that much impact ?
> Can you tell me how much memory both runs consume ?
> (It's often hard to tell true virtual memory consumption
> but real RAM consumption is quite a realistic indicator.)
>
> Do you have lots of hard links in your snapshots ?
> (It's only about inodes shared inside the directory
> trees which you map into xorriso. It does not matter
> whether there are other links pointing to those
> inodes from outside those directories.)
>
>
>> which was exceeded even with -temp_mem_limit 64k for some of the images.
>
> The impact of -temp_mem_limit is restricted to higher
> levels of xorriso, i fear. Things like pattern expansion
> and file list sorting by block address.
> If it works with setting 64k, then the memory consumption
> is probably inside libisofs.
>
>
>> The reason for setting -disk_dev_ino ino_only was for having the speed
>> advantage
>
> I was quite proud of that method when i developed it
> for scdbackup a few years before i enterd the libburnia
> adventure.
> It has to deal with the counter-intuitive fact that
> file timestamps do not change when the file gets
> renamed or moved to a different directory. Imagine
> two files swapping places.
> (Then imagine my face when i learned about Samba's
> inode dance.)
>
>
> Have a nice day :)
>
> Thomas
>
- Re: [Bug-xorriso] bug with handling symbolic links?, (continued)
- Re: [Bug-xorriso] bug with handling symbolic links?, Thomas Schmitt, 2015/02/16
- Re: [Bug-xorriso] bug with handling symbolic links?, Thomas Schmitt, 2015/02/21
- Re: [Bug-xorriso] bug with handling symbolic links?, Joerg Meyer, 2015/02/21
- Re: [Bug-xorriso] bug with handling symbolic links?, Thomas Schmitt, 2015/02/21
- Re: [Bug-xorriso] bug with handling symbolic links?, Thomas Schmitt, 2015/02/21
- Re: [Bug-xorriso] bug with handling symbolic links?, Joerg Meyer, 2015/02/22
Re: [Bug-xorriso] bug with handling symbolic links?, Thomas Schmitt, 2015/02/22
Re: [Bug-xorriso] bug with handling symbolic links?, Thomas Schmitt, 2015/02/22
Re: [Bug-xorriso] bug with handling symbolic links?, Thomas Schmitt, 2015/02/23