I've debugged the problem further and it appears the problem might not
be the JIT at all, but gnulib's putenv. I've used the MS appverifier
tool and it detected a heap corruption in gnulib's putenv
(re)implementation.
As far as I can see, the way it works when defining a new environment
variable is:
1) allocate a new char**, large enough to contain the current
environment + 1
2) copy the current environment in the newly allocated array
3) assign the new array to the POSIX "environ" variable
4) keep track of the newly created array into a static variable such
that it can be free'ed on the next envvar addition
And the problem lies in this last bit, if you link this code with some
3rd party code that calls the native CRT putenv implementation. The
native putenv implementation will do a realloc of the array created in
1), effectively trashing it. On the next gnulib's putenv call, at step
4), it will try to free the static pointer (stored during the previous
gnulib's putenv call), but that memory already has been de-allocated by
the native putenv call.
In practice in my case, octave first calls gnulib::putenv to add
OCTAVE_HOME, then my readline DLL calls crt::putenv to add LINES and
COLUMNS, then octave calls again gnulib::putenv to add GNUTERM. And
boom, it can crash on any further heap allocation.
Does the scenario above make sense? And what would be the correct fix?