Re: A vision for multiple major modes: some design notes

[Alan Mackenzie]

Hello, Drew.

On Wed, Apr 20, 2016 at 02:06:37PM -0700, Drew Adams wrote:
> Sounds very good, a priori.  And I commend you for actually
> putting together a clear and comprehensive design proposal
> for discussion, instead of just implementing something.
> Especially for something that is likely to lead to new uses
> and further possibilities, it is good to open up the big
> picture for discussion (regardless of the outcome).

Thanks, that's appreciated.

> Some feedback, mostly minor -

;-)

> >     * - For regexps which recognise whitespace, the regexp must contain
> >         "\\s-" or "\\s " or "[[:space:]]" so that the regexp engine will
> >         handle "foreign" islands and gaps between chained islands as 
> > whitespace.

> I understand the motivation (you explain it further on).  But this
> hardcoding of what can constitute a "whitespace-matching" pattern
> seems a bit rigid.  No way to flexibly allow for different meanings
> of whitespace here?  What if some code wants to handle \n or \t or
> \f etc. differently, or to even treat some set of (normally
> non-whitespace) chars as if they too were whitespace for island
> purposes?

This is a good point.  Maybe it would be better to match an island or
the gap between two chained islands with any regexp element which
matches the space (the good old 0x20 character).

> >   o - A @dfn{chain} of islands is a canonically ordered chain of islands in
> >       a single buffer.

> Why limit it necessarily to a single buffer?  It is common to
> want to do things (search etc.) across multiple buffers, and
> sometimes regardless of mode.  That doesn't diminish just
> because one might want to use chains of non-contiguous text
> zones.

Why limit it?  A buffer is a natural unit of editing.  The island chain
concept is primarily to allow different regions of a buffer to have
different major modes, whilst minimising ugly workarounds, artificial
restrictions, and so on.

> I'm pretty sure I would want to be able do things throughout
> a chain that spans different buffers.  If it were I, I would
> think about defining all that you are doing using a structure
> that is multi-buffer.

I don't envisage that the island chains will really be that useful for
(user initiated) searching, etc.  The idea is that, to the user, such a
buffer will look much like it already does, except that the font locking
will be appropriate for each island, the major mode key map will be
right for each island, and so on.

> [That is what I did for zones.el, for instance - sets of such
> text zones are delimited by markers, which automatically record
> the buffer they pertain to.  And they can be persistent, as well.
> Have you considered the possibility of persisting island chains?]

> And I would probably want user-level operations, to combine
> chains (append, intersect, union/coalesce, difference). 
> And why not be able to do that for chains that cross buffers?

The chains will be disjoint, so intersection/difference wouldn't be
useful.  Given that the essential feature of a chain is its major mode,
it wouldn't make sense to combine chains (which will usually have
different major modes).  I'm still trying to think through the idea of a
chain having islands in several buffers.

> Being able to add (e.g. append) a chain in one buffer to a chain
> in another buffer is one simple example.  Anything you might want
> to do with one chain you will likely want to be able to do with
> a set of chains, or at least with a chain that results from
> composing a set of chains in various ways.

> Also, I'm guessing/hoping, but I'm not sure I saw this explicitly,
> that you can have multiple chains (e.g. in the same buffer) that
> use the same major mode.

Indeed, yes.

> Being associated with a major mode is only one possible attribute of a
> chain - it is not required, and other attributes and uses of a chain
> are not dependent on it, right?  IOW, it is not necessary to think of
> chains as mode-related - that is just one (albeit common) use &
> interpretation, right?

Not right, sorry.  The major mode is an essential attribute of an island
chain.  There will be a slot for it in the structure which holds chain
data, just as there is currently a slot for it in the (C) buffer
structure.  There will likewise be slots for the syntax table, major
mode key map, and so on.  None of these slots would work well with a
null value.

> >   o - An island will be delimited in two complementary ways:
> >     * - It will be enclosed syntactically by characters with
> >       "open island" and "close island" syntax (see section (v)).
> >       Both of these syntactic markers will include a flag "chain"
> >       indicating whether there is a previous/next island in the
> >       chain.  The cdr of the syntax value will be
> >       the island chain to which the island belongs.
> >     * - It will be covered by the text property `island', whose
> >       value will be the pertinent island or island chain

> Are both always required, or is either sufficient for most
> purposes?

Both are required, yes.  They will both be used.

> Is the syntax one needed only when you need to take advantage of it?
> Can you do most things using either, so that a given operation (that
> is not specific to only one of them, e.g. not specific to syntax) can
> be done regardless of which is available?

Primarily, the text property is to allow the chain local variable
mechanism quickly to find the correct chain for accessing the variables
from.  There is a worry that the extra cost of accessing this text
property may slow Emacs down excessively.  There will probably have to
be some sort of cacheing of the current island.

> I'm thinking that in many contexts I would not care about
> delimiting by syntax, and I might not even care about
> associating a given chain with a mode.  Would I be able to
> use such chains nevertheless (e.g. search/replace across them)?

I'm not sure this island mechanism is the right tool for doing what
you're suggesting.  For searching/replacing at the user level, some
extra option meaning "only in the current chain" would need to be added
to the user interface.

> >       Note that if islands are enclosed inside other islands,

> Maybe you can elaborate on overlapping islands and chains? 
> What caveats or use cases do you see?

Islands would not be permitted (not sure how at this stage) to "overlap"
eachother.  Two islands must either be disjoint, or one completely
contain the other.  The major mode for any position would be that of the
"innermost" current island.

> A priori, I would like to have a chain data structure, and
> as much of the rest of the features as possible, be available
> and manipulable from Lisp.  Something like this has lots of
> enhancement possibilities and use cases that we are unlikely
> to imagine at the outset.  Implementing more than an absolute
> minimum in C hampers that exploration and improvement.

One idea would be to implement a chain feature, one of whose uses would
be the major mode islands I've been trying to specify.  A significant
part of this would have to be implemented at the C level for speed -
chain local variables are already going to be slower to access than
buffer local variables.  We must keep that difference to a minimum.

> HTH.  I don't claim to have grasped all of what you envisage.
> It's great food for thought, in any case.

> (I asked a couple of times, in the bug thread(s) and here,
> for just this sort of top-level picture of what was envisaged.
> I gave up hoping that someone might actually make clear what
> the question/project/plan is.  This is a welcome, if unexpected,
> development.)

Thanks!

-- 
Alan Mackenzie (Nuremberg, Germany).