SEGV initializing embedded interpreter inside a dynamic shared object ?

[Pierre Baldensperger]

Hello all,

I am trying to revive a rather old "embedded octave" program from 2005 
(according to the accompanying readme, it was developed for Octave 
2.1.50). It is an OpenDX add-on module allowing to call octave scripts 
from OpenDX Visual Programs and have them operate on OpenDX arrays and 
fields (neat functionality btw).

More specifically, this program is the "OctaveOp" module from the 
"CISM_DX" project, and was written by Robert S. Weigel (School of 
Physics Astronomy and Computational Science, George Mason University, 
Fairfax, Virginia). The original source code is available from the 
CISM_DX website and can be found in the spdx/src/calloctave subdirectory 
after unpacking.

I have tried to adapt the source code and Makefile to have it compile 
against Octave 3.8.0 (extensive API changes wrt Octave 2.1), OpenDX 
4.4.4 and g++ 4.8 on OpenSuSE 13.1, but I'm now stuck with an obscure 
problem right at the initialization of the Octave interpreter (call to 
octave_main) : it triggers a segmentation violation downstream the 
octave_process_command_line function.

This initialization is one of the parts I had to replace entirely to 
account for the new API. I have followed the guidelines for standalone 
applications from the manual (and they indeed work perfectly for 
building a standalone embedded octave program with "mkoctfile 
--link-stand-alone"). Here's how octave_main is called (this is straight 
from the manual, nothing different) :


------------------
  (...)

  #include <octave/oct.h>
  #include <octave/octave.h>
  #include <octave/parse.h>
  #include <octave/toplev.h>

  (...)

  string_vector argv (2);
  argv(0) = "embedded";
  argv(1) = "--silent";
  octave_main (2, argv.c_str_vec (), true);

  (...)

  clean_up_and_exit(0);
------------------



The tricky thing here is that this embedded interpreter is eventually 
not initialized straight from the main() function of a standalone 
program, but from a deep function call located inside a DSO (OpenDX 
external module) that is loaded at run-time by the OpenDX server process 
"dxexec". Following the OpenDX developer manual, I have managed to 
attach a gdb debugging session to "dxexec", which apparently caught and 
narrowed the SEGV location. Here's the backtrace:



------------------
(...)
OctaveOp.c was passed the command OctaveOpDemoValueList(ValueList,'hello 
world'); from OpenDX.
A 2 item, 6-vector list passed to OctaveOp.c from OpenDX:
2001.000000 1.000000 1.000000 0.000000 0.000000 0.000000
2000.000000 1.000000 1.000000 0.000000 0.000000 0.000000


Program received signal SIGSEGV, Segmentation fault.
0x00007ffff664c1ce in _getopt_internal_r (argc=2, argv=0xb3e8b0, 
optstring=0x7ffff1509c4c "HWVdfhip:qvx", longopts=0x7ffff1a37720 
<long_opts>,
    longind=0x7fffffffcc0c, address@hidden, 
address@hidden <getopt_data>, 
address@hidden) at getopt.c:462
462     getopt.c: Aucun fichier ou dossier de ce type.


(gdb) where
#0  0x00007ffff664c1ce in _getopt_internal_r (argc=2, argv=0xb3e8b0, 
optstring=0x7ffff1509c4c "HWVdfhip:qvx", longopts=0x7ffff1a37720 
<long_opts>,
    longind=0x7fffffffcc0c, address@hidden, 
address@hidden <getopt_data>, 
address@hidden) at getopt.c:462

#1  0x00007ffff664d18b in _getopt_internal (argc=<optimized out>, 
argv=<optimized out>, optstring=<optimized out>, longopts=<optimized out>,
    longind=<optimized out>, address@hidden, 
address@hidden) at getopt.c:1175

#2  0x00007ffff664d213 in getopt_long (argc=<optimized out>, 
argv=<optimized out>, options=<optimized out>, long_options=<optimized 
out>,
    opt_index=<optimized out>) at getopt1.c:65

#3  0x00007ffff0c8f7bb in octave_process_command_line 
(address@hidden, address@hidden) at octave.cc:533

#4  0x00007ffff0c92c31 in octave_main (argc=2, argv=0xb3e8b0, 
embedded=1) at octave.cc:512

#5  0x00007ffff1a46b12 in OctaveOp (command=0xc6c7b0 
"OctaveOpDemoValueList(ValueList,'hello world');", 
OctOpStrct=0x7ffff1c4e3e0 <m_OctaveOp::OctOpStrct>,
    verbosity=2) at OctaveOp.cpp:40

#6  0x00007ffff1a4b4dd in m_OctaveOp (in=0xc6df20, out=0xc6df40) at 
OctaveOp.c:287

#7  0x000000000044079f in _execGnode (ptr=<optimized out>) at 
evalgraph.c:1415

#8  0x00000000004581cd in _dxf_ExRQDequeue (address@hidden) at rq.c:443

#9  0x000000000043a722 in ExCheckRunqueue (graphId=0) at dxmain.c:2202

#10 ExMainLoopMaster () at dxmain.c:675

#11 ExMainLoop () at dxmain.c:538

#12 ExChildProcess () at dxmain.c:530

#13 DXmain (argc=<optimized out>, argv=<optimized out>, envp=<optimized 
out>) at dxmain.c:434

#14 0x00007ffff659ebe5 in __libc_start_main (main=0x430a80 <main>, 
argc=4, argv=0x7fffffffd7c8, init=<optimized out>, fini=<optimized out>,
    rtld_fini=<optimized out>, stack_end=0x7fffffffd7b8) at 
libc-start.c:269

#15 0x0000000000430ad5 in _start () at ../sysdeps/x86_64/start.S:122
------------------


From the verbose debug messages at the beginning, it seems that the 
external DSO has indeed been loaded and is working as expected inside 
the dxexec process, and that the data transmission from the OpenDX UI 
went fine. From my limited knowledge and debugging skills, I tried to 
check the input arguments but found nothing anomalous:


------------------
(gdb) up
#1  0x00007ffff664d18b in _getopt_internal (argc=<optimized out>, 
argv=<optimized out>, optstring=<optimized out>, longopts=<optimized out>,
    longind=<optimized out>, address@hidden, 
address@hidden) at getopt.c:1175
1175    in getopt.c

(gdb) up
#2  0x00007ffff664d213 in getopt_long (argc=<optimized out>, 
argv=<optimized out>, options=<optimized out>, long_options=<optimized 
out>,
    opt_index=<optimized out>) at getopt1.c:65
65      getopt1.c: Aucun fichier ou dossier de ce type.

(gdb) up
#3  0x00007ffff0c8f7bb in octave_process_command_line 
(address@hidden, address@hidden) at octave.cc:533
533     octave.cc: Aucun fichier ou dossier de ce type.

(gdb) p argc
$1 = 2

(gdb) p argv[0]
$2 = 0xb30710 "embedded"

(gdb) p argv[1]
$3 = 0xb30730 "--silent"
------------------


For what is worth, the offending line at octave.cc:533 is the following:

------------------
(...)
int optc = getopt_long (argc, argv, short_opts, long_opts, &long_idx);
(...)
------------------


Trying to look at the other arguments in this function call:


------------------
(gdb) p short_opts
$5 = <optimized out>

(gdb) p long_opts
$4 = {{name = 0x7ffff1509ceb "braindead", has_arg = 0, flag = 0x0, val = 
19}, {name = 0x7ffff1509cf5 "built-in-docstrings-file", has_arg = 1, 
flag = 0x0,
    val = 1}, {name = 0x7ffff1509d0e "debug", has_arg = 0, flag = 0x0, 
val = 100}, {name = 0x7ffff1509d14 "debug-jit", has_arg = 0, flag = 0x0, 
val = 9}, {
    name = 0x7ffff1509d1e "doc-cache-file", has_arg = 1, flag = 0x0, 
val = 2}, {name = 0x7ffff1509d2d "echo-commands", has_arg = 0, flag = 
0x0, val = 120},
  {name = 0x7ffff1513ba6 "eval", has_arg = 1, flag = 0x0, val = 3}, 
{name = 0x7ffff1509d3b "exec-path", has_arg = 1, flag = 0x0, val = 4}, {
    name = 0x7ffff1509d45 "force-gui", has_arg = 0, flag = 0x0, val = 
5}, {name = 0x7ffff1509d4f "help", has_arg = 0, flag = 0x0, val = 104}, {
    name = 0x7ffff1509d54 "image-path", has_arg = 1, flag = 0x0, val = 
6}, {name = 0x7ffff1509d5f "info-file", has_arg = 1, flag = 0x0, val = 7}, {
    name = 0x7ffff1509d69 "info-program", has_arg = 1, flag = 0x0, val 
= 8}, {name = 0x7ffff1509d76 "interactive", has_arg = 0, flag = 0x0, val 
= 105}, {
    name = 0x7ffff1509d82 "jit-compiler", has_arg = 0, flag = 0x0, val 
= 10}, {name = 0x7ffff1509dbe "line-editing", has_arg = 0, flag = 0x0, 
val = 11}, {
    name = 0x7ffff1509d8f "no-gui", has_arg = 0, flag = 0x0, val = 12}, 
{name = 0x7ffff1509d96 "no-history", has_arg = 0, flag = 0x0, val = 72}, {
    name = 0x7ffff1509da1 "no-init-file", has_arg = 0, flag = 0x0, val 
= 13}, {name = 0x7ffff1509dae "no-init-path", has_arg = 0, flag = 0x0, 
val = 14}, {
    name = 0x7ffff1509dbb "no-line-editing", has_arg = 0, flag = 0x0, 
val = 15}, {name = 0x7ffff1509dcb "no-site-file", has_arg = 0, flag = 
0x0, val = 16},
  {name = 0x7ffff1509dd8 "no-window-system", has_arg = 0, flag = 0x0, 
val = 87}, {name = 0x7ffff1509de9 "norc", has_arg = 0, flag = 0x0, val = 
102}, {
    name = 0x7ffff15a5c82 "path", has_arg = 1, flag = 0x0, val = 112}, 
{name = 0x7ffff1509dee "persist", has_arg = 0, flag = 0x0, val = 17}, {
    name = 0x7ffff1509df6 "quiet", has_arg = 0, flag = 0x0, val = 113}, 
{name = 0x7ffff1509dfc "silent", has_arg = 0, flag = 0x0, val = 113}, {
    name = 0x7ffff1509e03 "texi-macros-file", has_arg = 1, flag = 0x0, 
val = 18}, {name = 0x7ffff1509e14 "traditional", has_arg = 0, flag = 
0x0, val = 19},
  {name = 0x7ffff15a5df4 "verbose", has_arg = 0, flag = 0x0, val = 86}, 
{name = 0x7ffff15b8419 "version", has_arg = 0, flag = 0x0, val = 118}, 
{name = 0x0,
    has_arg = 0, flag = 0x0, val = 0}}

(gdb) p long_idx
$6 = 0
------------------


Has anybody already encountered a similar situation before? Maybe I miss 
something obvious, or is there some subtle difference for initializing 
an embedded interpreter inside a DSO?

While searching the web for more info about this, I noticed that several 
people apparently managed to embed an interpreter in a Java JNI (which 
also entails putting the API calls inside a DSO that is loaded at 
run-time by the JVM, so not entirely unlike the OpenDX external add-on 
mechanism). There may be some similar procedure that I could try to 
follow here, if someone can provide a good pointer to one such JNI 
project I could examine.

Thank you in advance for any hint as to what may be happening or what I 
could do to progress towards solving this problem.

-Pierre.