RE: [Gcl-devel] memory damaged at(system:STRING-MATCH:anykey:verbose) - PS

[Mike Thomas]

Hi Camm.

Thanks for the reply.  I've included a diff to the relevant parts so you can
see what I'm doing and attached "sfasl.c" in it's full glory as that will be
easier to understand than the diff.

| OK, from the list below, I still think we might only need rdata.  I
| think the other 'data' sections refer to c++.  We need only worry
| about sections with relocatable objects therein, so .stabs et. al. are
| not a problem.

Agreed - I'm ignoring .stabs etc, although I would like to see a C++ GCL one
day.

One thing which concerns me is the ordering of the sections.

Our code assumes you can just assign an index and always get the right
section eg RDATA_NSCN etc (with a subsidiary check on the section name which
doesn't trigger a further search if it fails).  However I can't find any
documentation to suggest that the order is fixed, so perhaps a loop through
the sections might be the go if the ordering really is not fixed.  (Here I'm
trying also to leave a window open for the freely available version of
Visual C++ which may be a viable alternative to gcc on Windows.)  Also I am
thinking about just reading those sections required into memory rather than
sucking up everything (stabs and all) as I am doing in the attached code.

One also wonders, on the side, whether the stabs information might not be
useful for mixed Lisp/C level debugging?


| > Yes, except for the GPL rather than LGPL licencing which I'm
| still trying to
| > minimise and the apparent? divergence of our BFD usage from the modern
| > recommended method, which leaves Windows GCL in an unsupported
| part of the
| > BFD library.
|
| Interesting.  And to me the biggest difficulty was the 1.5Meg extra
| memory consumption!

I have mixed feelings about the memory consumption debate and in general
prefer not to worry about it.

The only times I have ever really thought it crucial for acceptability of
GCL were when I wondered about the possibility of an openMosix cluster
situation, where parallelism is fork/exec/spawn driven and processes migrate
over the network, or if someone was to build a fork/exec/spawn server of
some sort.


|  The license issue always seemed insignificant to
| me given our current and likely future user base.

It's not fatal for me obviously, but it is a consideration - I would like to
minimise obstacles to some future worker making GCL do for Lisp what GCC did
for C/C++ or, more selfishly, to myself if I decide to write a commercial
app in CL (unlikely admittedly - I would almost certainly use some other
language).


| Great!  Please let me know if you get stuck or need another pair of
| eyes.  Just figuring out the proper value of the relocation in this
| example, and stepping through in gdb to make sure that is handled
| correctly, will likely solve the whole thing.

Yes, alright, having ignored the voice of reason about BFD  (my own issues
with that path aside) I've been stuck since Friday!!.  Maxima builds but
falls over on startup.  if you agree with the assumptions I've made below
and the way I've implemented them in the attached diffs and code, then it
will be time to break out gdb again I suppose.

I got a bit further after I worked out that the .rdata section relocations
weren't happening because of a misdefinition in the NTYPE/N_SECTION macros
(can't remember which it was off hand).

My approach has been:

1. Define NSCN and string macros for .rdata, .stabs and .stabstr.

2. Assume that .rdata relocations in the relocate() function (rel_coff.c)
are exactly the same as .data relocations except that the value of
"the_start" is set to sfaslp->s_start_rdata, which is in turn set as
follows:

        sfaslp->s_start_data  = start_address + textsize;
        sfaslp->s_start_bss   = start_address + textsize + datasize;
      sfaslp->s_start_rdata = start_address + textsize + datasize
                            + bsssize + stabsize + stabstrsize;

3. That in relocate_symbols(), RDATA_NSCN is treated the same way as
DATA_NSCN except that n_value is set to sfaslp->s_start_rdata as follows:

      n_value = (int)sfaslp->s_start_rdata;

I haven't worked out the implications of the alignment of .rdata which is
apparently 2**2 rather than 2**4 for .data as shown below.


If you agree that all these assumptions are correct I can only assume I've
blundered somewhere and hopefully you eye run over the code might pick up
something obvious.

Here is skeletal information from objdump on "clmacs.o".

$ objdump  -h src/binary-gcl/clmacs.o

src/binary-gcl/clmacs.o:     file format pe-i386

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00007310  00000000  00000000  00000104  2**4
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
  1 .data         000003c0  00000000  00000000  00007414  2**4
                  CONTENTS, ALLOC, LOAD, RELOC, DATA
  2 .bss          00000000  00000000  00000000  00000000  2**4
                  ALLOC
  3 .stab         0000861c  00000000  00000000  000077d4  2**2
                  CONTENTS, RELOC, READONLY, DEBUGGING
  4 .stabstr      00005776  00000000  00000000  0000fdf0  2**0
                  CONTENTS, READONLY, DEBUGGING
  5 .rdata        0000003c  00000000  00000000  00015566  2**2
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA



===================================================================
RCS file: /cvsroot/gcl/gcl/h/ext_sym.h,v
retrieving revision 1.4
diff -r1.4 ext_sym.h
13a14,16
> #define STAB_NSCN 4
> #define STABSTR_NSCN 5
> #define RDATA_NSCN 6

===================================================================
RCS file: /cvsroot/gcl/gcl/h/minglacks.h,v
retrieving revision 1.1
diff -r1.1 minglacks.h
27a28,30
> #define NTYPE(sym) (sym)->n_type)
> #define N_TYPE(sym) (sym)->n_type)
> #define N_SECTION(sym) (sym->n_section)

===================================================================
RCS file: /cvsroot/gcl/gcl/h/coff/i386.h,v
retrieving revision 1.4
diff -r1.4 i386.h
105,109c105,112
< #define _TEXT ".text"
< #define _DATA ".data"
< #define _BSS  ".bss"
< #define _COMMENT ".comment"
< #define _LIB ".lib"
---
> #define _TEXT     ".text"
> #define _DATA     ".data"
> #define _BSS      ".bss"
> #define _STAB     ".stab"
> #define _STABSTR    ".stabstr"
> #define _RDATA            ".rdata"
> #define _COMMENT    ".comment"
> #define _LIB        ".lib"

===================================================================
RCS file: /cvsroot/gcl/gcl/o/sfasl.c,v
retrieving revision 1.19
diff -r1.19 sfasl.c
58d57
<
78a78
>     char *s_start_rdata;
173c173
<     int textsize, datasize, bsssize, nsyms;
---
>     int textsize, datasize, bsssize, stabsize, stabstrsize, rdatasize,
nsyms;
183a184
>     size_t sections_read = 0;
188a190,192
>
>     memset ( section, 0, 10 * sizeof ( struct scnhdr ) );
>
214,217c218,226
<     fread ( &section[1],
<             fileheader.f_nscns,
<             sizeof ( struct  scnhdr ),
<             fp );
---
>     fprintf ( stderr, "fasload: Attempting to read %d sections\n",
fileheader.f_nscns );
>     sections_read = fread ( &section[1],
>                             sizeof ( struct  scnhdr ),
>                             fileheader.f_nscns,
>                             fp );
>     if ( sections_read != fileheader.f_nscns ) {
>         fprintf ( stderr, "Could not get the expected number of sections
(%d), got %d\n", fileheader.f_nscns, sections_read );
>     }
>
220c229,230
<     if (strcmp(section[BSS_NSCN].s_name, ".bss") == 0) {
---
>
>     if (strcmp(section[BSS_NSCN].s_name, _BSS) == 0) {
224a235,253
>
>     if (strcmp(section[STAB_NSCN].s_name, _STAB) == 0) {
>         stabsize=section[STAB_NSCN].s_size;
>     } else {
>         stabsize=section[STAB_NSCN].s_size = 0;
>     }
>
>     if (strcmp(section[STABSTR_NSCN].s_name, _STABSTR) == 0) {
>         stabstrsize=section[STABSTR_NSCN].s_size;
>     } else {
>         stabstrsize=section[STABSTR_NSCN].s_size = 0;
>     }
>
>     if ( strcmp ( section[RDATA_NSCN].s_name, _RDATA ) == 0 ) {
>         rdatasize = section[RDATA_NSCN].s_size;
>         fprintf ( stderr, "fasload: .rdata section found of %d bytes, %d
relocations\n", rdatasize, section[RDATA_NSCN].s_nreloc );
>     } else {
>         rdatasize = section[RDATA_NSCN].s_size = 0;
>     }
302c331
<                                     datasize+textsize+bsssize,
---
>
datasize+textsize+bsssize+stabsize+stabstrsize+rdatasize,
313c342
<             memory->cfd.cfd_size = datasize + textsize + bsssize +
extra_bss;
---
>             memory->cfd.cfd_size = datasize + textsize + bsssize +
stabsize + stabstrsize + rdatasize + extra_bss;
318,319c347,349
<             sfaslp->s_start_data = start_address + textsize;
<             sfaslp->s_start_bss = start_address + textsize + datasize;
---
>             sfaslp->s_start_data  = start_address + textsize;
>             sfaslp->s_start_bss   = start_address + textsize + datasize;
>             sfaslp->s_start_rdata = start_address + textsize + datasize +
bsssize + stabsize + stabstrsize;
326c356
<             = malloc ( datasize + textsize + bsssize + extra_bss +
0x80000 );
---
>             = malloc ( datasize + textsize + bsssize + stabsize +
stabstrsize + rdatasize + extra_bss + 0x80000 );
330c360
<             = malloc ( datasize + textsize + bsssize + extra_bss );
---
>             = malloc ( datasize + textsize + bsssize + stabsize +
stabstrsize + rdatasize + extra_bss );
332,333c362,364
<       sfaslp->s_start_data = start_address + textsize;
<       sfaslp->s_start_bss = start_address + textsize + datasize;
---
>       sfaslp->s_start_data  = start_address + textsize;
>       sfaslp->s_start_bss   = start_address + textsize + datasize;
>         sfaslp->s_start_rdata = start_address + textsize + datasize +
bsssize + stabsize + stabstrsize;
335c366
<       dprintf(" Code size %d, ", datasize+textsize+bsssize + extra_bss);
---
>       dprintf(" Code size %d, ", datasize+rdatasize+textsize+bsssize +
extra_bss);
339,340c370,371
<       SAFE_FREAD ( the_start, textsize + datasize, 1, fp );
<       dprintf("read %d bytes of text + data into memory at ", textsize +
datasize );
---
>       SAFE_FREAD ( the_start, textsize + datasize + bsssize + stabsize +
stabstrsize + rdatasize, 1, fp );
>       dprintf("read %d bytes of text + data into memory at ", textsize +
datasize + bsssize + stabsize + stabstrsize + rdatasize );
377,388c408,428
<             for ( j = 1; j < BSS_NSCN ; j++) {
<                 dprintf("relocating section %d \n",j);
<                 if (section[j].s_nreloc) fseek(fp,section[j].s_relptr,0);
< #  ifdef ADJUST_RELOC_START
<                 ADJUST_RELOC_START(j);
< #  endif
<                 for ( i=0; i < section[j].s_nreloc; i++ ) {
<                     /* RELSZ = sizeof(relocation_info) */
<                     fread ( &relocation_info, RELSZ, 1, fp);
<                     dprintf ( "    item %3d: ", i );
<                     relocate() ;
<                 }
---
>             for ( j = 1; j <= RDATA_NSCN ; j++ ) {
>                 if ( ( j != BSS_NSCN ) && ( j != STAB_NSCN ) &&
>                      ( j != STABSTR_NSCN ) && ( 0 !=
section[j].s_nreloc ) ) {
>                     dprintf ( "relocating section %d \n", j );
>                     fprintf ( stderr, "relocating section %d, %d
relocations \n",
>                               j, section[j].s_nreloc );
>                     fseek ( fp, section[j].s_relptr, 0 );
>                     switch ( j ) {
>                     case TEXT_NSCN:  the_start = memory->cfd.cfd_start;
break;
>                     case DATA_NSCN:  the_start = sfaslp->s_start_data;
break;
>                     case RDATA_NSCN: the_start = sfaslp->s_start_rdata;
break;
>                     default:
>                         the_start = memory->cfd.cfd_start;
>                         fprintf ( stderr, "fasload: Uncatered section
relocation %d.\n", j );
>                     }
>                     for ( i=0; i < section[j].s_nreloc; i++ ) {
>                         fread ( &relocation_info, RELSZ, 1, fp );
>                         dprintf ( "    item %3d: ", i );
>                         relocate() ;
>                     }
>                 }
423a464,466
>         dprintf(" stabsize is %x\n",stabsize);
>         dprintf(" stabstrsize is %x\n",stabstrsize);
>         dprintf(" rdatasize is %x\n",rdatasize);
476a520
> #if 0
481c525
<             fwrite(start_address,sizeof(char),datasize+textsize,out);
---
>
fwrite(start_address,sizeof(char),datasize+textsize+stabsize+stabstrsize+rda
tasize,out);
483a528
> #endif
628c673
<         case TEXT_NSCN : case DATA_NSCN: case BSS_NSCN :
---
>         case TEXT_NSCN : case DATA_NSCN: case RDATA_NSCN: case BSS_NSCN :
631a677,680
>         if (typ==RDATA_NSCN) {
>               fprintf ( stderr, "RDATA_NSCN value setting %x, %s\n",
(int)sfaslp->s_start_rdata, SYM_NAME(sym) );
>               n_value = (int)sfaslp->s_start_rdata;
>           }
647a697,699
>             case RDATA_NSCN:
>                 fprintf ( stderr, "(RDATA_NSCN)");
>                 break;
672a725
>           fprintf ( stderr, "relocate_symbols: am ignoring a scnum %d\n",
(sym->n_scnum) );

sfasl.c
Description: Binary data