Re: [igraph] Apparent memory corruption in R

[Tamas Nepusz]

Hi Luka,

I have managed to reproduce the problem on my machine with R version
3.5.2 and igraph installed with install.packages('igraph"). In my
case, the error message looked slightly different, though:

Error in head.default("2623", c(1L, 1L)) : length(n) == 1L is not TRUE
Calls: lapply ... contract.vertices -> head -> head.default -> stopifnot
R(33662,0x1198925c0) malloc: *** error for object 0x7fb6eddc8720:
pointer being freed was not allocated
R(33662,0x1198925c0) malloc: *** set a breakpoint in malloc_error_break to debug

I am not the developer of the R interface so I am not really familiar
with the internals -- plus, a large chunk of the glue code between R
and the C core of igraph is auto-generated so some things may go wrong
there. I suspect a double-free issue that happens when
contract.vertices() (from the C layer) calls back into R code to
perform the merging of some vertex or edge attributes during
contractions.

Judging from the call to "head" I also suspect that the crash is
somehow related to the usage of "first" in vertex.attr.comb=... as
"first" requires a call to head() internally. My theory is that the
call to head() in the vertex attribute combination phase throws an
error (this is what we see above), which accidentally triggers igraph
to double-free something (and that things would be solved if I managed
to fix that error). I tried to confirm this by removing "first" from
vertex.attr.comb=... to see if that would resolve the problem, but
unfortunately the code stops with the following error message:

Error in findInterval(t$pos, vec = compress$pos) :
  'vec' must be sorted non-decreasingly and not contain NAs

As I don't know what findInterval() is supposed to do, I'm a bit at a
loss here. Can you help me with this?

All the best,


T.

On Sat, 19 Jan 2019 at 01:23, Luka Culibrk <address@hidden> wrote:
>
> Hi Tamas,
>
> Thank you for your reply. I've packaged the data required to replicate the 
> crash and a script which replicates the crash on my machine. It can be 
> downloaded from 
> https://drive.google.com/file/d/1csjRW-ftIYmJdI_LdadUCHOk9-Tz1btp/view?usp=sharing
>  .
> This can be run with:
>
> tar -xvf igraph_debug.tar.gz; cd igraph_debug; Rscript debug_igraph_crash.R
>
> Required packages are dplyr and igraph.
>
> Thanks,
> Luka
>
> On Thu, Jan 17, 2019 at 1:45 AM Tamas Nepusz <address@hidden> wrote:
>>
>> Dear Luka,
>>
>> Unfortunately we can't really help without seeing a minimal,
>> reproducible example code that causes the crash. Here are some
>> instructions on providing such an example that we can start debugging
>> on our side:
>>
>> https://stackoverflow.com/a/5963610/156771
>>
>> All the best,
>> Tamás
>>
>> On Tue, 15 Jan 2019 at 21:57, Luka Culibrk <address@hidden> wrote:
>> >
>> > Dearest igraph users,
>> >
>> > I'm running into some issues with igraph in R.
>> >
>> > I've posted this same issue to Stack Overflow 
>> > (https://stackoverflow.com/posts/54155455) and was recommeded to come here.
>> >
>> > Context: I have a path graph that I wish to contract iteratively; ie. at 
>> > each iteration, take a set of adjacent vertices in the path graph that fit 
>> > some critera, contract them together, and repeat iteratively until no 
>> > remaining vertices fit the criteria. Each path graph represents a 
>> > biological chromosome (this is biological data that I'm working with). The 
>> > path graph to contract is roughly 5,000 vertices long, and the program 
>> > iterates roughly 20 times at most per graph. Each sample case has 22 path 
>> > graphs to contract. Prototyping via Rstudio worked just fine (and still 
>> > works fine), however when running this as an Rscript it's throwing some 
>> > errors, and in a different iteration each time (ie. this seems to be 
>> > happening at random)
>> >
>> > The first time I encountered the issue, it would throw a double free 
>> > error. Again, this would occur unpredictably. I tried running on a 
>> > different machine, however this threw a different memory-related error 
>> > which I don't remember. I made a new miniconda virtual environment and 
>> > reinstalled all of the requisite R packages, including igraph and tried 
>> > again. Now I'm getting another error, here:
>> > ```
>> > Error in head.default("2616", 1L) : length(n) == 1L is not TRUE
>> > Calls: write.table ... contract.vertices -> head -> head.default -> 
>> > stopifnot
>> > In addition: There were 28 warnings (use warnings() to see them)
>> > Error in contract.vertices(toy, mapping = vertices, vertex.attr.comb = 
>> > list(new_residual = "sum",  :
>> >   VECTOR_ELT() can only be applied to a 'list', not a 'double'
>> > Calls: write.table ... compressdata -> %>% -> eval -> eval -> 
>> > contract.vertices -> .Call
>> > Execution halted
>> > ```
>> > Note that in "head.default("2616", 1L)", the number (here "2616") is 
>> > always different every time I run the code. I have no idea what is 
>> > actually calling head.default() since I don't see it anywhere in the 
>> > contract.vertices() source code that I can get using 
>> > edit(getAnywhere(contract.vertices())). I've also scoured the C source 
>> > code on github which hasn't gotten me anywhere.
>> >
>> > I'm running this on CentOS 7 machines with 1tb of memory each.
>> >
>> > Here is the double free error with a few lines of the memory dump, note 
>> > that the ```Error: length(n) == 1L is not TRUE``` is being thrown inside 
>> > an igraph function (```contract_vertices()```) rather than my script. I 
>> > see this error when I run it from a different anaconda environment that I 
>> > have for another project.
>> >
>> > ```
>> > Error: length(n) == 1L is not TRUE
>> > In addition: There were 29 warnings (use warnings() to see them)
>> > *** glibc detected *** 
>> > /home/lculibrk/miniconda3/envs/lukonda/lib/R/bin/exec/R: double free or 
>> > corruption (out): 0x00007ffffb3ea4a0 ***
>> > ======= Backtrace: =========
>> > /lib64/libc.so.6[0x3e9f275f4e]
>> > /lib64/libc.so.6[0x3e9f278cf0]
>> > /home/lculibrk/R/x86_64-pc-linux-gnu-library/3.4/igraph/libs/igraph.so(igraph_vector_destroy+0x27)[0x7f8ccac45337]
>> > /home/lculibrk/R/x86_64-pc-linux-gnu-library/3.4/igraph/libs/igraph.so(IGRAPH_FINALLY_FREE+0x63)[0x7f8ccab5b24d]
>> > /home/lculibrk/R/x86_64-pc-linux-gnu-library/3.4/igraph/libs/igraph.so(R_igraph_finalizer+0xc1)[0x7f8ccac90976]
>> > ```
>> >
>> > The consistent thing about the double free error above is it seems to 
>> > occur in the same memory address each time; 0x00007ffffb3ea4a0. Still, it 
>> > occurs in a different iteration each time, and on different path graphs 
>> > each time.
>> >
>> > I'm quite lost as to where exactly I can start the process of trying to 
>> > debug this issue, as I'm sure it must be on my end somewhere.
>> >
>> > Thanks for any help you might be able to provide. Let me know if I need to 
>> > clarify anything or provide more information.
>> >
>> > Best regards,
>> > Luka Culibrk
>> >
>> > _______________________________________________
>> > igraph-help mailing list
>> > address@hidden
>> > https://lists.nongnu.org/mailman/listinfo/igraph-help
>>
>> _______________________________________________
>> igraph-help mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/igraph-help
>
> _______________________________________________
> igraph-help mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/igraph-help