Re: using finalizers

[Tomas Hlavaty]

On Sun 02 Jan 2022 at 08:54, Eli Zaretskii <eliz@gnu.org> wrote:
> One common paradigm for such use cases is to use a callback: the
> function that traverses some collection of objects calls that callback
> for every object it finds.

There is no such thing in directory-files.

Also using callbacks in directory-files alone would not fix
find-lisp-find-dired.  I think an extra execution thread would be
needed.  Maybe using an extra execution thread could be seen as a work
around for the negative consequences of stack discipline.  If there was
a callback in directory-files, it would allow for pushing directory
items out of the execution thread when encountered, instead of waiting
until the contents of the whole directory has been collected.

> There are usually protocols for the callback to stop the process and
> prevent the rest of objects from being examined/reported.

There is no such thing in directory-files.

I can't see such thing in any other mapping functions in Emacs Lisp.
Could you point me to an example?

Also, such protocols are non-trivial and usually complicate the code
significantly.  See for example how people deal with it JavaScript,
where they have no other choice.

>> But when should close be called, when such stream is lazy?  I am
>> sure a reasonable ad-hoc solution could be implemented.  But what if
>> the directory is traversed recursively?  When should each close be
>> called?  A case for another ad-hoc solution?  Looks like finalizers
>> would be great here.
>
> We have the unwind-protect machinery in place for that.  In fact, if
> you look at the implementation of directory-files, you will see that
> it already uses that machinery, and it most probably actually kicked
> in when you typed C-g above.

I see I did not manage to get the point across.  directory-files is
"eager" and follows stack discipline so unwind-protect there makes
sense.  But the idea of directory-files itself is broken as I
demonstrated previously.  Operating systems get it right and provide
streaming API with open/read/close directory functions.  Also the
brokenness of directory-files propagates to many other points in Emacs
that need to list directories, like find-lisp-find-dired and completion.

But the point here was that in case of lazy sequences, the point in time
where close should be called is not known upfront.  One would have to
split the code into two parts: eager one which would force stack
discipline and lazy one which would do the lazy part.  Maybe doable for
trivial thing like lazy listing of a single directory, but it gets
harder when doing something non-trivial like lazy listing of a directory
recursively.