Re: Changes to lexer invalidate documentation

[Rik]

John W. Eaton wrote:
> On 13-Apr-2009, Rik wrote:
>
> | What I imagined is that the sequence of inputs was parsed and as each
> | logical group of commands was divided the result was placed in a queue. 
> | After parsing a simple while (!empty) loop grabs the head item from the
> | queue and passes it to the evaluation engine.  This architecture has a
> | strict division between parsing, which occurs at one time, and
> | evaluation, which occurs at a different time.  I was imagining that it
> | would be simple to intervene when a single block of commands was
> | available at the head of the queue.  Instead of finishing all of the
> | parsing the engine would evaluate the first group 'A' and only if that
> | succeeded would it move to parse B.
> | 
> | The more I think about it the more pathological the example in the
> | documentation becomes.  First, I find it really rare to enter long
> | multiple commands on a single line.  I would normally code a function on
> | the command line like so:
> | 
> | octave:1> functon y = f (x)<CR>
> | > y = x^2;<CR>
> | > endfunction<CR>
> | 
> | In this case the first line WOULD be evaluated by itself and produce an
> | error before I ever got to lines 2 and 3.  Secondly, it has to be just
> | the right sort of parsing error to pass initial parsing checks but fail
> | a subsequent one.  Most of the time I have a much simpler typo which the
> | parser will catch.  As an example, the current parser correctly catches
> | a failure to close a parenthesis and indicates where.
> | 
> | a = sin(pi; b = cos(pi)<CR>
> | parse error:
> | 
> |   syntax error
> | 
> | >>> a = sin(pi; b = cos(pi)
> |               ^
>
> Yes, but this is a case where the syntax error occurs in the first
> expression in the list.
>   
The point I meant to imply here was that very few inputs are
pathological (either they are ordinary typos or the line structure is
not sufficient to trigger an error) and that statistically the parser
will correctly identify the location of most errors.  I still think the
more desirable behavior is to direct the user to the root error, a
misspelling of 'function', but I don't think it rises to the level of
requiring a patch when there are more worthy things to code.
> | In view of the rarity of the documentation example (and the fact that
> | I'm changing it to something simpler) it is probably not worth bothering
> | about.
> | > I don't see how we can change that behavior.  Imagine typing
> | >
> | >   missing_function (); x+++
> | >
> | > and having your typing interrupted by an error message about the
> | > missing function as you started to type "x+++", or typing
> | >
> | >   y = rand (10), x = 1
> | >
> | > and getting output (the display of Y) before you finished typing "x = 1".
> | >   
> | I wasn't thinking of a just-in-time parser that was processing
> | character-by-character.  The current parser is engaged by the carriage
> | return character and then works on the entire line.  There is no
> | intermediate output from one command obscuring another because the
> | semicolon is specifically there for silencing output.
>
> But semicolons are not the only command separator.
>   
But they are the only separator for which I was proposing a change.  All
of my thinking derives from the example in the documentation and I
wouldn't change the behavior for the comma separator precisely because
of the difficulty in determining when to begin printing output.

--Rik