[Bug-mit-scheme] [bug #19505] IMAIL silently fails when message header fields exceed available memory

[Taylor R. Campbell]

URL:
  <http://savannah.gnu.org/bugs/?19505>

                 Summary: IMAIL silently fails when message header fields
exceed available memory
                 Project: MIT/GNU Scheme
            Submitted by: riastradh
            Submitted on: Tuesday 04/03/07 at 21:11
                Category: imail
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect behavior
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                Keywords: 

    _______________________________________________________

Details:

In a sufficiently large folder (which is not very large -- the one I was
trying this with has about fifteen hundred messages in it), when preloading
the folder's outlines exhausts the available heap space, IMAIL silently
fails, and sometimes disconnects from the server.  I'm not sure exactly where
or when it is disconnecting from the server -- when I try to sort a folder, it
doesn't disconnect, but when I try to summarize the folder, it does
disconnect.  If IMAIL does not disconnect, the heap space remains nearly
exhausted until I manually disconnect, which is sometimes impossible with the
amount of space left.

Some possible ways to work around this come to mind:

1. Store each message header fields string in an external string until
parsing it is requested, almost like file folders do, only with more overhead
(one external string per message, rather than one per folder and an interval
of it per message).
2. Add a secondary GC daemon to de-initialize message header fields, or store
weak references to them.
3. Don't preload cached items; load those on demand.

The latter two require, which could operate conjointly, changing the method
for PRELOAD-FOLDER-OUTLINES on IMAP folders to answer each FETCH response
incrementally: currently all the responses are stored in-core collected in a
list until control returns to PRELOAD-FOLDER-OUTLINES to process the list.

With the third option, it might be better to make the header fields be a
stream, rather than a list, so that it would be unnecessary to parse the
whole header every time the cached item is loaded.  On the other hand, the
headers may not be large enough for this to matter much.

In the mean time, it would also be nice if IMAIL reported the heap exhaustion
in some way, rather than silently failing and either disconnecting or possibly
locking up Edwin.  The only way I can think now to implement this would be to
create the editor cmdl with the console I/O port rather than a dummy I/O
port, although I'm a little confused about how failure while generating a
summary buffer manages to close the folder first.




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?19505>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/