Re: using finalizers

[Eli Zaretskii]

> From: Tomas Hlavaty <tom@logand.com>
> Cc: rudi@constantly.at, emacs-devel@gnu.org
> Date: Sun, 02 Jan 2022 08:53:31 +0100
> 
> On Sun 02 Jan 2022 at 08:54, Eli Zaretskii <eliz@gnu.org> wrote:
> > One common paradigm for such use cases is to use a callback: the
> > function that traverses some collection of objects calls that callback
> > for every object it finds.
> 
> There is no such thing in directory-files.

It should be easy to add, if we wanted to.  Or we could implement a
separate set of primitives using that, if it were deemed useful.

I fail to see how what's in Emacs now is of any essence for the
purposes of this discussion.  What exactly are you getting at, and how
is what is in Emacs today relevant?  Are you saying that Emacs
couldn't possibly provide such primitives due to some inherent
limitation?  If so, please explain why you think so, because I don't
think I agree.

> Also using callbacks in directory-files alone would not fix
> find-lisp-find-dired.

You'd need a different implementation of that, of course.  Other than
that, what is your point?

> I think an extra execution thread would be needed.

I don't see why.  A callback can easily insert files into the buffer,
and trigger redisplay either directly or via timers and such.  Keep in
mind that when Emacs invokes an async subprocess, the display of what
that subprocess produces is still done in the same single thread where
the Lisp machine runs.  So we already have the capabilities of
"pretending" we have multithreading, when these aspects are involved.

> > There are usually protocols for the callback to stop the process and
> > prevent the rest of objects from being examined/reported.
> 
> There is no such thing in directory-files.

I think we have something like that in lisp/url/.

> Also, such protocols are non-trivial and usually complicate the code
> significantly.

I don't see why this would be non-trivial.  In the simplest case, the
callback returns a special value which tells to stop the iteration.

> > We have the unwind-protect machinery in place for that.  In fact, if
> > you look at the implementation of directory-files, you will see that
> > it already uses that machinery, and it most probably actually kicked
> > in when you typed C-g above.
> 
> I see I did not manage to get the point across.  directory-files is
> "eager" and follows stack discipline so unwind-protect there makes
> sense.  But the idea of directory-files itself is broken as I
> demonstrated previously.  Operating systems get it right and provide
> streaming API with open/read/close directory functions.  Also the
> brokenness of directory-files propagates to many other points in Emacs
> that need to list directories, like find-lisp-find-dired and completion.

Emacs is not an OS (the popular myth about that notwithstanding).  It
provides primitives that are useful for writing editing and
text-processing applications.  What you don't find in Emacs is
probably outside of its scope, or at least is rarely if ever needed.

> But the point here was that in case of lazy sequences, the point in time
> where close should be called is not known upfront.  One would have to
> split the code into two parts: eager one which would force stack
> discipline and lazy one which would do the lazy part.  Maybe doable for
> trivial thing like lazy listing of a single directory, but it gets
> harder when doing something non-trivial like lazy listing of a directory
> recursively.

Where you see something non-trivial, I see just simple use of our
unwinding infrastructure.  There's no problem I see with using that
infrastructure in recursive processing of directories.  If you see
such problems, please tell what they are, specifically, given the
unwind_protect functions we have and the mechanism that actually
implements the unwinding.  In Emacs, many operations could signal an
error or throw, and the user could type C-g at any moment, so this
mechanism is in use all the time, and must handle correctly nested
unwinders.