Re: [Ifile-dev] Adding some extra switches

[Jason Rennie]

address@hidden said:
> However, ifile doesn't seem to like being given large numbers of files
> (tens of thousands) to process in one go. The kernel eventually killed
> it after trashing VM for a while. I didn't have time to dig deeply
> into why. I'll have to come back to it when I have more time. 

Jeremy Brown ripped out some of the bloat that made ifile not handle lots
of messages very well in a patch for 1.1.0, but it looks like there may
still be a leak somewhere (I regularly do runs w/ up to 2000 messages and
see memory usage increase during the run---which shouldn't happen if ifile
is sequentially processing the messages and freeing memory as it goes).

One question---how do your patches get around this problem?  Seems like
ifile would suck up all the memory whether you pass files via the
command-line or via stdin...

Sorry for all the pestering---just don't want to be adding cmd line 
options too freely :)

Jason D. M. Rennie
MIT AI Lab
address@hidden
(617) 253-5339
http://www.ai.mit.edu/~jrennie/