Re: The current state of the comment-cache branch

[Alan Mackenzie]

Hello, Eli.

On Wed, Dec 28, 2016 at 05:35:47PM +0200, Eli Zaretskii wrote:
> > Date: Tue, 27 Dec 2016 16:40:18 +0000
> > Cc: address@hidden
> > From: Alan Mackenzie <address@hidden>

> > Hello, Eli.

> > On Sat, Dec 24, 2016 at 02:27:26PM +0200, Eli Zaretskii wrote:
> > > > Date: Sat, 24 Dec 2016 09:42:46 +0000
> > > > From: Alan Mackenzie <address@hidden>
> > > > Cc: address@hidden

> > > > > Can you show similar timings with that variable set to nil?  And in
> > > > > particular in the use case reported by Paul back when bug#22884 was
> > > > > filed?

> > > > master:                        101.939s            95.049s        
> > > > 103.546s
> > > > (with open-paren-... nil)

> > > Hmm... doesn't match my timings here.

> > > With an optimized (-Og) build of Emacs 25.1.90, I get 42.844 with
> > > open-paren-in-column-0-is-defun-start t and 62.234 nil.  With the
> > > unoptimized build of the current master, I get 193.922 and 1117.469,
> > > respectively.  The 193 sec result is marginally reasonable for an
> > > unoptimized build, the 1117 sec result is IMO preposterous, and the
> > > 6-fold slowdown is IMO too much to be explained just by the variable.

Might your unoptimized build have included the extra checking code which
can be enabled?  My debugging build didn't.

> > If the buffer is large enough (were you using xdisp.c?) a large factor
> > slowdown from using o-p-i-c-0-i-d-s seems inevitable.

> I was using the same file xdisp.c and the same timing code as you.
> That's why I don't understand how come our results are so different.
> With such differences unexplained, how can we ever trust these
> benchmarks?

Well, they are qualitative, and somewhat quantitative.  They are enough
to say "quite a bit slower" with, even if they can't tell us "exactly
44.5% slower".

> > > Anyway, is your suggestion to adopt the code on that branch and set
> > > open-paren-in-column-0-is-defun-start to nil?  If we are not going to
> > > set it to nil, then the speed advantage is IMO too small to justify
> > > the changes.

> > No.  My suggestion is that open-paren-in-column-0-is-defun start ceases
> > to play any role in syntax.c.

> Does that mean to effectively set it to nil?  Otherwise, I don't
> understand how can we judge the current situation vs what you suggest.

The code in comment-cache does back_comment by a totally different
algorithm which simply doesn't involve
open-paren-in-column-0-is-defun-start.  It is as fast as open-... being
set to t, yet as accurate (or more so) than open-... being set to nil.

> > Instead comments and strings will be scanned only in the forward
> > direction, the results of these scan operations being cached in a
> > text property.

> Btw, adding text properties slows down redisplay.  I don't think the
> slow-down is significant, but it's worth having in mind anyway.

I don't think it's significant, either.  time-scroll redisplays the
entire buffer, and we don't see a slow-down there.

> > > Also, I wonder why do we need all these changes in syntax.c, when the
> > > problem is AFAIK only with C mode and its derivatives.  Are these
> > > changes beneficial in any way to modes with non-C syntax?

> > The changes are in syntax.c because this is where the fundamental cause
> > of the problem lies.  I don't believe it is possible to solve them in CC
> > Mode.  I don't know why the problem doesn't manifest itself in other
> > modes.  Perhaps other modes do less scanning of comments backwards (with
> > `scan-sexps' with a negative count).

Sorry, that scan-sexps should have been scan-lists.

> If we don't completely understand the problem, and cannot explain why
> it happens only in CC mode, then how do we make sure we solve the
> problem which needs to be solved?

I think I do understand the problem fully, and I think Stefan does, too.

> Why would the amount of back-scanning through comments depend on the
> major mode?  I don't think that C programmers write more/longer
> comments than programmers in any other programming language.

It might be that functions like c-parse-state (which does a lot of
scan-lists in the backward direction) are more prominent in CC Mode.
Could it possibly be that most other languages don't have such large
buffers as C, C++, ...?  It might be that wrongly recognising an open
paren as a BOD does less damage to syntax analysis in other languages.
Or it could be that apostrophes are used a lot in comments, and in CC
Mode modes, "'" has string syntax - an odd number of apostrophes in a
comment will then trigger the code which involves open-paren-in-....

Or, it could be a combination of all these things.

-- 
Alan Mackenzie (Nuremberg, Germany).