Re: rec-mode in Emacs for large files slow (+90k records, 57MB)

[Jose E. Marchesi]

Hello.

> I’m using the recfiles format for a large database of around 97k records
> in a 57MB file.  Browsing records in Emacs with rec-mode is fine for the
> first few, but at the end of the file it becomes very slow.
>
> For example, `rec-cmd-goto-previous-rec` takes 16 seconds to go from the
> last record to the second to last.  I ran a profile of it and I think
> that the issue is that `rec-beginning-of-field-pos` is slow at the end
> of the file for some reason.  Viz.
>
> ```
>      Samples    %   Function  
>         1734  99% - command-execute
>         1734  99%  - funcall-interactively
>         1531  88%   - rec-cmd-goto-previous-rec
>          966  55%    - rec-goto-previous-rec
>          964  55%     - rec-beginning-of-record
>          964  55%      - rec-beginning-of-record-pos
>          960  55%         rec-beginning-of-field-pos
>          230  13%    - rec-record-type
>          230  13%     - rec-current-record-descriptor
>          230  13%      - rec-current-record
>          228  13%       - rec-beginning-of-record-pos
>          227  13%          rec-beginning-of-field-pos
>            2   0%       + rec-parse-record
>          215  12%    - rec-show-record
>          214  12%     - rec-narrow-to-record
>          122   7%      + rec-beginning-of-record-pos
>           92   5%      + rec-end-of-record-pos
>            1   0%     + rec-update-mode-line
>          120   6%    + rec-current-record
>          203  11%   + counsel-M-x
>            4   0% + redisplay_internal (C function)
>            0   0% + ...
> ```
>
> Does anyone have any ideas on speeding this up?  If I disable rec-mode,
> Emacs has no problem moving page by page at the end of the file: it’s
> not so big that it hits the big file problem yet.
>
> Apologies if this is the wrong forum.  I’m not sure where else to ask.

I just pushed a patch that changes rec-beginning-of-field-pos to not use
the looking-back function, which can become very slow when invoked in
big buffers.

Moving across records in a big file shouldn't be now more slow than in a
small file.

Thanks for reporting.