gensub crash

[Miguel Pineiro Jr.]

Hello, everyone. I hope this report finds you well.

While working on ota's gensub, I discovered that gawk's gensub crashes when 
there is no match and the final argument is a string converted from the result 
of a binary arithmetic operation with at least one variable operand.

$ git branch --verbose --list master
* master c2999b43 Another small doc fix.
$ ./gawk 'BEGIN { x=1; print gensub(/a/, "b", 1, x+1) }'
gawk: cmd. line:1: fatal error: internal error
Aborted (core dumped)

All of the following also dump core: x+1, x-1, x*1, x/1, x^1. But if there's a 
match (even if it's not selected), all work fine (and valgrind reports no 
invalid reads).

$ ./gawk 'BEGIN { x=1; print gensub(/2/, "two", 1, x+1) }'
two
$ ./gawk 'BEGIN { x=1; print gensub(/2/, "two", 2, x+1) }'
2


In case it's useful, gawk 5.1.1 on Fedora 36 doesn't crash. It produces 
gibberish and the amount of gibberish equals the length of the final argument's 
string.

$ gawk --version | head -n1
GNU Awk 5.1.1, API: 3.1 (GNU MPFR 4.1.0-p13, GNU MP 6.2.1)
$  gawk 'BEGIN { x=1; print gensub(/a/, "b", 1, x+1) }' | hexdump -C
00000000  e8 0a                                             |..|
00000002
$  gawk 'BEGIN { x=1; print gensub(/a/, "b", 1, x+12345) }' | hexdump -C
00000000  70 1b 0c ef b6 0a                                 |p.....|
00000006
$  gawk 'BEGIN { x=1; print gensub(/a/, "b", 1, x+123456789) }' | hexdump -C
00000000  f6 7d 79 0d 17 56 00 00  00 0a                    |.}y..V....|
0000000a


Here's what valgrind produces for the current master branch (c2999b43):
 
$ valgrind --leak-check=full ./gawk 'BEGIN { x=1; print gensub(/a/, "b", 1, 
x+1) }'
==399909== Memcheck, a memory error detector
==399909== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==399909== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info
==399909== Command: ./gawk BEGIN\ {\ x=1;\ print\ gensub(/a/,\ "b",\ 1,\ x+1)\ }
==399909== 
==399909== Invalid read of size 8
==399909==    at 0x41F02F: do_sub (builtin.c:3334)
==399909==    by 0x43E5E7: r_interpret (interpret.h:1136)
==399909==    by 0x4064F6: main (main.c:549)
==399909==  Address 0x4b81640 is 16 bytes inside a block of size 88 free'd
==399909==    at 0x48480E4: free (vg_replace_malloc.c:872)
==399909==    by 0x41F44C: DEREF (awk.h:1297)
==399909==    by 0x41F44C: do_sub (builtin.c:3333)
==399909==    by 0x43E5E7: r_interpret (interpret.h:1136)
==399909==    by 0x4064F6: main (main.c:549)
==399909==  Block was alloc'd at
==399909==    at 0x484586F: malloc (vg_replace_malloc.c:381)
==399909==    by 0x459675: emalloc_real (awk.h:2064)
==399909==    by 0x459675: r_getblock (node.c:1150)
==399909==    by 0x459E71: make_number_node (awk.h:2114)
==399909==    by 0x459E71: r_make_number (node.c:370)
==399909==    by 0x43DD44: r_interpret (interpret.h:605)
==399909==    by 0x4064F6: main (main.c:549)
==399909== 
==399909== Invalid read of size 8
==399909==    at 0x41F033: do_sub (builtin.c:3334)
==399909==    by 0x43E5E7: r_interpret (interpret.h:1136)
==399909==    by 0x4064F6: main (main.c:549)
==399909==  Address 0x4b81638 is 8 bytes inside a block of size 88 free'd
==399909==    at 0x48480E4: free (vg_replace_malloc.c:872)
==399909==    by 0x41F44C: DEREF (awk.h:1297)
==399909==    by 0x41F44C: do_sub (builtin.c:3333)
==399909==    by 0x43E5E7: r_interpret (interpret.h:1136)
==399909==    by 0x4064F6: main (main.c:549)
==399909==  Block was alloc'd at
==399909==    at 0x484586F: malloc (vg_replace_malloc.c:381)
==399909==    by 0x459675: emalloc_real (awk.h:2064)
==399909==    by 0x459675: r_getblock (node.c:1150)
==399909==    by 0x459E71: make_number_node (awk.h:2114)
==399909==    by 0x459E71: r_make_number (node.c:370)
==399909==    by 0x43DD44: r_interpret (interpret.h:605)
==399909==    by 0x4064F6: main (main.c:549)
==399909== 
==399909== Invalid read of size 1
==399909==    at 0x484C9A0: memcpy@GLIBC_2.2.5 (vg_replace_strmem.c:1121)
==399909==    by 0x459A8B: make_str_node (node.c:423)
==399909==    by 0x43E5E7: r_interpret (interpret.h:1136)
==399909==    by 0x4064F6: main (main.c:549)
==399909==  Address 0x4b816d0 is 0 bytes inside a block of size 2 free'd
==399909==    at 0x48480E4: free (vg_replace_malloc.c:872)
==399909==    by 0x459048: r_unref (node.c:542)
==399909==    by 0x41F44C: DEREF (awk.h:1297)
==399909==    by 0x41F44C: do_sub (builtin.c:3333)
==399909==    by 0x43E5E7: r_interpret (interpret.h:1136)
==399909==    by 0x4064F6: main (main.c:549)
==399909==  Block was alloc'd at
==399909==    at 0x484586F: malloc (vg_replace_malloc.c:381)
==399909==    by 0x458EC1: emalloc_real (awk.h:2064)
==399909==    by 0x458EC1: r_format_val.part.0 (node.c:300)
==399909==    by 0x41E111: force_string_fmt (awk.h:1978)
==399909==    by 0x41E111: force_string_fmt (awk.h:1962)
==399909==    by 0x41E111: do_sub (builtin.c:3011)
==399909==    by 0x43E5E7: r_interpret (interpret.h:1136)
==399909==    by 0x4064F6: main (main.c:549)
==399909== 
2
==399909== 
==399909== HEAP SUMMARY:
==399909==     in use at exit: 82,317 bytes in 1,010 blocks
==399909==   total heap usage: 1,525 allocs, 515 frees, 114,815 bytes allocated
==399909== 
==399909== LEAK SUMMARY:
==399909==    definitely lost: 0 bytes in 0 blocks
==399909==    indirectly lost: 0 bytes in 0 blocks
==399909==      possibly lost: 0 bytes in 0 blocks
==399909==    still reachable: 82,317 bytes in 1,010 blocks
==399909==         suppressed: 0 bytes in 0 blocks
==399909== Reachable blocks (those to which a pointer was found) are not shown.
==399909== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==399909== 
==399909== For lists of detected and suppressed errors, rerun with: -s
==399909== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)


For the system's gawk 5.1.1:

$ valgrind --leak-check=full gawk 'BEGIN { x=1; print gensub(/a/, "b", 1, x+1) 
}'
==399874== Memcheck, a memory error detector
==399874== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==399874== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info
==399874== Command: gawk BEGIN\ {\ x=1;\ print\ gensub(/a/,\ "b",\ 1,\ x+1)\ }
==399874== 
==399874== Invalid read of size 1
==399874==    at 0x484C9A0: memcpy@GLIBC_2.2.5 (vg_replace_strmem.c:1121)
==399874==    by 0x173BC3: UnknownInlinedFun (string_fortified.h:29)
==399874==    by 0x173BC3: make_str_node (node.c:416)
==399874==    by 0x14FE21: r_interpret (interpret.h:1067)
==399874==    by 0x117472: main (main.c:526)
==399874==  Address 0x4d5dfe0 is 0 bytes inside a block of size 2 free'd
==399874==    at 0x48480E4: free (vg_replace_malloc.c:872)
==399874==    by 0x171E40: r_unref (node.c:511)
==399874==    by 0x1383A2: UnknownInlinedFun (awk.h:1288)
==399874==    by 0x1383A2: do_sub (builtin.c:3200)
==399874==    by 0x14FE21: r_interpret (interpret.h:1067)
==399874==    by 0x117472: main (main.c:526)
==399874==  Block was alloc'd at
==399874==    at 0x484586F: malloc (vg_replace_malloc.c:381)
==399874==    by 0x17453C: UnknownInlinedFun (awk.h:2014)
==399874==    by 0x17453C: r_format_val.part.0 (node.c:292)
==399874==    by 0x13739E: UnknownInlinedFun (awk.h:1931)
==399874==    by 0x13739E: do_sub (builtin.c:2898)
==399874==    by 0x14FE21: r_interpret (interpret.h:1067)
==399874==    by 0x117472: main (main.c:526)
==399874== 
2
==399874== 
==399874== HEAP SUMMARY:
==399874==     in use at exit: 106,861 bytes in 401 blocks
==399874==   total heap usage: 471 allocs, 70 frees, 135,205 bytes allocated
==399874== 
==399874== LEAK SUMMARY:
==399874==    definitely lost: 0 bytes in 0 blocks
==399874==    indirectly lost: 0 bytes in 0 blocks
==399874==      possibly lost: 0 bytes in 0 blocks
==399874==    still reachable: 106,861 bytes in 401 blocks
==399874==         suppressed: 0 bytes in 0 blocks
==399874== Reachable blocks (those to which a pointer was found) are not shown.
==399874== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==399874== 
==399874== For lists of detected and suppressed errors, rerun with: -s
==399874== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

If this goes unfixed, I may eventually look into it myself. But at the moment, 
ota is consuming most of my spare time.

Take care,
Miguel