Re: State of the overlay tree branch?

[Sebastian Sturm]

after disabling cquery for testing purposes (which leaves me with 45 
overlays generated by flycheck for my troublesome C++ file), I'm now 
generating a large number of overlays using the following function:

(defun test-highlight ()
        (save-excursion
          (with-silent-modifications
            (let ((stepsize 10))
              (widen)
              (goto-char 1)
              (cl-loop for n from (point-min) upto (- (point-max) 
stepsize) by stepsize do
                (let ((ov (make-overlay n (+ (1- stepsize) n))))
                  (overlay-put ov 'cquery-sem-highlight t)))))))

Evaluating the following function without additional overlays (beyond 
the flycheck ones, that is) yields the following results:

(defun benchmark-often ()
  (cl-loop for n from 1 upto 20 do
           (message (format "iteration %d: %f" n (nth 0 (benchmark-run 
(line-number-at-pos (point))))))))

1st run:
iteration 1: 0.001213
iteration 2: 0.001170
iteration 3: 0.001170
iteration 4: 0.001238
iteration 5: 0.001163
iteration 6: 0.001153
iteration 7: 0.000421
iteration 8: 0.000426
iteration 9: 0.000322
iteration 10: 0.000301
iteration 11: 0.000291
iteration 12: 0.000292
iteration 13: 0.000291
iteration 14: 0.000291
iteration 15: 0.000295
iteration 16: 0.000289
iteration 17: 0.000289
iteration 18: 0.000288
iteration 19: 0.000288
iteration 20: 0.000287

2nd run:
iteration 1: 0.001044
iteration 2: 0.000942
iteration 3: 0.000935
iteration 4: 0.000935
iteration 5: 0.000935
iteration 6: 0.000932
iteration 7: 0.000954
iteration 8: 0.000940
iteration 9: 0.000933
iteration 10: 0.000625
iteration 11: 0.000545
iteration 12: 0.000428
iteration 13: 0.000362
iteration 14: 0.000346
iteration 15: 0.000325
iteration 16: 0.000309
iteration 17: 0.000309
iteration 18: 0.000316
iteration 19: 0.000310
iteration 20: 0.000308

after evaluating (test-highlight) the figures are as follows:
1st run:
iteration 1: 0.026012
iteration 2: 0.020334
iteration 3: 0.020250
iteration 4: 0.020349
iteration 5: 0.020501
iteration 6: 0.020635
iteration 7: 0.020302
iteration 8: 0.020426
iteration 9: 0.020440
iteration 10: 0.020441
iteration 11: 0.020515
iteration 12: 0.020525
iteration 13: 0.020383
iteration 14: 0.020510
iteration 15: 0.019829
iteration 16: 0.019899
iteration 17: 0.019950
iteration 18: 0.019828
iteration 19: 0.019901
iteration 20: 0.019819

2nd run:
iteration 1: 0.026526
iteration 2: 0.020051
iteration 3: 0.020100
iteration 4: 0.020080
iteration 5: 0.020080
iteration 6: 0.020249
iteration 7: 0.020087
iteration 8: 0.020005
iteration 9: 0.019980
iteration 10: 0.019985
iteration 11: 0.020077
iteration 12: 0.019979
iteration 13: 0.020060
iteration 14: 0.020092
iteration 15: 0.019954
iteration 16: 0.019766
iteration 17: 0.019432
iteration 18: 0.019491
iteration 19: 0.019458
iteration 20: 0.019482

I'm not allowed to share my employer's source code as a test case, so I 
tried the same procedure with the similarly large DeclBase.h from the 
public LLVM repository. To my surprise, DeclBase.h didn't suffer from 
any performance issues at all. Switching to fundamental-mode while 
visiting my file didn't change anything, so I assume that c-mode isn't 
to blame either.
There have been claims of overlay-related performance issues with cquery 
and some large(-ish) open-source C or C++ files, so I'll try to locate 
these files and hope that at least one of them exhibits this issue as well.

On 03/19/2018 04:07 PM, Sebastian Sturm wrote:
> for the file I was complaining about, the number returned is 3080 
> (doesn't exceed the number of chars though, (point-max) = 67641). Will 
> try to obtain usable timing data later today when I get home from work. 
> Thanks!
>
> On 03/19/2018 03:56 PM, Stefan Monnier wrote:
> > > well no, it's about 2.5ms per call to line-number-at-pos, which is 
> > > called at
> > > least 6 times per character insertion (with my Emacs config, at
> > > least). Which already makes for 15ms per character insertion, excluding
> > > anything else done by cc-mode or lsp-mode.
> >
> > Since you say that the noverlay branch helps, could you check the number
> > of overlays involved?  E.g.
> >
> >      M-: (length (overlays-in (point-min) (point-max))) RET
> >
> > If there are more overlays than chars in this buffer, maybe there's
> > a problem in some Elisp that creates too many overlays?
> >
> > If there aren't that many overlays, then I wonder why the noverlays
> > branch would make such a significant difference.
> >
> > Also, if you can reliably reproduce the "slow editing", would it be
> > possible to make a recipe for it that we can try and reproduce on
> > our side?
> >
> >
> >          Stefan