Re: Mixed EOL characters causes improper font locking and often wisi parser hangs

[Stephen Leake]

brownt.fully@aleeas.com writes:

> I've encountered situations where there are multiple developers and
> sometimes (likely due to merging), we will end up with source files
> that contain both CRLF and LF line endings on different lines of the
> source file.  Ideally, these should be cleaned up and only a single
> line ending should be used for a file, however it is a reality which
> likely will not be immediately solved.

Ok.

> When files with mixed line endings are encountered in ada-mode, I've
> seen a couple different problems. The least severe is that the font
> locking ends up being skewed and certain characters are colored in the
> wrong font.

Yes; emacs treats the extra CR as characters in the file, but the lexer
treats them as part of the new_line sequence, so the character positions
differ.

It should be possible to tell the lexer to work the way Emacs does; I'll
put that on my list. I doubt the change will be simple enough to provide
a patch for wisi 3.1.1.

> In the worst case scenario, it appears the back-end wisi parser hangs
> and is not able to successfully process the source file, leaving the
> entire file without any font locking.

In my current working version (ie beta test), with your test below, the
byte count on the command line sent from emacs to the process is wrong,
so the next command gets mangled and the process aborts.

And then when I try to edit the file, the parser mysteriously hangs.

> This has become problematic enough that I've actually created my own
> macro to clean-up these files when I run across them, save the file
> and then kill off the wisi parser (via wisi-parse-kill) and restart
> ada-mode.

I think we should include that code in ada-mode, if I can't fix the
lexer properly.

> I'm not exactly sure why the parser hangs in certain scenarios and not
> in others, but I suspect that it just depends on where the
> interleaving of the line endings occurs and how complicated the source
> is to parse.

It also depends on the sequence of commands - if one command has the
wrong length, the next one gets mangled.

> The following simple hello world program can demonstrate the skewed
> font locking.

Thanks for the simple test case; I'll add it to my tests.

--
-- Stephe