Re: [gforth] ORDER bug

[Anton Ertl]

I have now created a hybrid heuristic test and tested it.  It
recognizes fewer names (3158), and the (revised) test against the new
(current) HEAD? shows:

[~/gforth:76432] gforth xxx.fs -e "test cr bye"|grep "hybrid accepts"|wc -l
142
[~/gforth:76433] gforth xxx.fs -e "test cr bye"|grep "new accepts"|wc -l
622

Looking at the output of this test, it appears that there are quite a
number of false negatives coming out of the new HEAD?:

[~/gforth:76434] gforth xxx.fs -e "test cr bye"|grep -B1 recurse        
hybrid accepts as head: $7FFFF6BFA7E0 len=7 
7FFFF6BFA7F0: 72 65 63 75  72 73 65    -                           recurse
[~/gforth:76435] gforth xxx.fs -e "test cr bye"|grep -B1 cfalign
hybrid accepts as head: $7FFFF6BF3020 len=9 
7FFFF6BF3030: 63 66 61 6C  69 67 6E 65 - 64                        cfaligned
--
hybrid accepts as head: $7FFFF6BF9D58 len=7 
7FFFF6BF9D68: 63 66 61 6C  69 67 6E    -                           cfalign

Or, more relevantly:

s" recurse" find-name .s <1> 140737333143520  ok
head? . 0  ok

It seems that it does not recognize aliases.

I have tried to put the hybrid version into int.fs and test it, but
the kernel does not work then (this only becomes visible on the next
round of kernel-building).  I probably did something wrong wrt cross.fs.

I have now also prepared .VOC to report ??? for unsafe guesses of the
old and hybrid versions.  The problem is actually a general problem of
the combination of heuristic HEAD? and .VOC as currently implemented:
A false positive can lead to NAME>INT being called for an
inappropriate address, and NAME>INT will segfault.  There's a test of
the CFA at the end of NEW-HEAD?, but is it guaranteed to catch all
cases?

- anton

Here's the code:

\ test head? implementations

: new-head? ( addr -- f )
\G heuristic check whether addr is a name token; may deliver false
\G positives; addr must be a valid address
    dup dup aligned <>
    if
        drop false exit \ heads are aligned
    then
    name>string dup $20 $1 within if
        2drop false exit \ realistically the name is short
    then
    + cfaligned @ here forthstart within ; \ and the cfa is outside

: old-head? ( addr -- f )
\G heuristic check whether addr is a name token; may deliver false
\G positives; addr must be a valid address; returns 1 for
\G particularly unsafe positives
    \ we follow the link fields and check for plausibility; two
    \ iterations should catch most false addresses: on the first
    \ iteration, we may get an xt, on the second a code address (or
    \ some code), which is typically not in the dictionary.
    \ we added a third iteration for working with code and ;code words.
    3 0 do
        dup dup aligned <> if \ protect @ against unaligned accesses
            drop false unloop exit
        then
        dup @ dup
        if ( addr addr1 )
            dup rot forthstart within
            if \ addr1 is outside forthstart..addr, not a head
                drop false unloop exit
            then ( addr1 )
        else \ 0 in the link field, no further checks
            2drop 1 unloop exit \ this is very unsure, so return 1
        then
    loop
    \ in dubio pro:
    drop true ;

: hybrid-head? ( addr -- f )
\G heuristic check whether addr is a name token; may deliver false
\G positives; addr must be a valid address; returns 1 for
\G particularly unsafe positives
    \ we follow the link fields and check for plausibility; two
    \ iterations should catch most false addresses: on the first
    \ iteration, we may get an xt, on the second a code address (or
    \ some code), which is typically not in the dictionary.
    \ we added a third iteration for working with code and ;code words.
    3 0 do
        dup dup aligned <> if \ protect @ against unaligned accesses
            drop false unloop exit
        then
        dup name>string nip $20 $1 within if
            drop false unloop exit \ realistically the name is short
        then
        dup @ dup if ( addr addr1 )
            dup rot forthstart within
            if \ addr1 is outside forthstart..addr, not a head
                drop false unloop exit
            then ( addr1 )
        else \ 0 in the link field, no further checks
            2drop 1 unloop exit \ this is very unsure, so return 1
        then
    loop
    \ in dubio pro:
    drop true ;

: my-.name ( nt -- )
    name>string dup ." len=" .
    16 min dump ;

: test
    here forthstart do
        i new-head? 0<> i hybrid-head? 0<> over <> if
            cr if
                ." new"
            else
                ." hybrid"
            then
            ."  accepts as head: " i hex. i my-.name
        else
            drop
        then
    loop ;

: testhybrid
    0 here forthstart do
        i hybrid-head? 0<> -
    loop ;