guile-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

GC Warning related to large mem block allocation - Help needed


From: David Pirotte
Subject: GC Warning related to large mem block allocation - Help needed
Date: Fri, 29 Dec 2017 23:48:21 -0200

Hello,

Currently, using guile-cv upon large images triggers these warnings,
using guile guile 2.2.3:

        GC Warning: Repeated allocation of very large block (appr. size 
775237632):
        May lead to memory leak and poor performance.

        ... then 1 '=' above msg per block allocation, tens to hundreds, 
sometimes
        even thousands of these, depending on the image processing being done 
...

These are totally unnecessary (I know I'm allocating these blocks), renders the
repl 'output' unreadable/unusable and actually scare users (well, I only have 
one
user for now).

So I am trying to understand and solve this problem. Below a (naive) attempt to
patch guile so it uses GC_malloc_ignore_off_page for objects > 100Kb, but that
did not even work: I guess I do miss most if not all of the puzzle pieces 
here...

While I wrote this tiny patch,  2 quiz also raised in my mind, here there are,
together with the naive patch (below):

1-    should this function return volatile ? (and how)

In bwd-gc README file you can read:

        https://github.com/ivmai/bdwgc

GC_malloc_ignore_off_page(bytes)

    Identical to GC_malloc, but the client promises to keep a pointer to the
    somewhere within the first 256 bytes of the object while it is live. (This
    pointer should normally be declared volatile to prevent interference from
    compiler optimizations.) This is the recommended way to allocate anything 
that
    is likely to be larger than 100 Kbytes or so. (GC_malloc may result in 
failure
    to reclaim such objects.)

and if yes, how would I achieve this since this function has multiple return
calls (with different pointer type) ?

2-    I don't understand the last return 'case'

    ...
      return scm_inline_gc_alloc
        (&thread->freelists[idx], idx, SCM_INLINE_GC_KIND_NORMAL);
    ...

How should that be patched so it also use GC_malloc_ignore_off_page(bytes)
when bytes > 100Kb?

Thanks,
David

;;;
;;; Below the patch that does not work :)
;;;

From 0ae4f18e478b2e390de72f560ff8428edfeda860 Mon Sep 17 00:00:00 2001
From: David PIROTTE <address@hidden>
Date: Fri, 29 Dec 2017 22:40:52 -0200
Subject: [PATCH] Use GC_malloc_ignore_off_page for objects > 100Kb

* libguile/gc-inline.h (scm_inline_gc_malloc): Use
  GC_malloc_ignore_off_page for objects > 100Kb.
---
 libguile/gc-inline.h | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/libguile/gc-inline.h b/libguile/gc-inline.h
index fcbe5a54e..7d02e67cc 100644
--- a/libguile/gc-inline.h
+++ b/libguile/gc-inline.h
@@ -119,8 +119,12 @@ scm_inline_gc_malloc (scm_i_thread *thread, size_t bytes)
 {
   size_t idx = scm_inline_gc_bytes_to_freelist_index (bytes);
 
-  if (SCM_UNLIKELY (idx >= SCM_INLINE_GC_FREELIST_COUNT))
-    return GC_malloc (bytes);
+  if (SCM_UNLIKELY (idx >= SCM_INLINE_GC_FREELIST_COUNT)) {
+    if (bytes > 100000)
+      return  GC_malloc_ignore_off_page (bytes);
+    else
+      return GC_malloc (bytes);
+  }
 
   return scm_inline_gc_alloc
     (&thread->freelists[idx], idx, SCM_INLINE_GC_KIND_NORMAL);
-- 
2.15.0

Attachment: pgpVxCugdGiAG.pgp
Description: OpenPGP digital signature


reply via email to

[Prev in Thread] Current Thread [Next in Thread]