Re: Identifying sources of allocations in a toy Mandelbrot package

[Psionic K]

Most importantly to my question, how can I identify sources of consing
without just knowing which functions and macros pathologically cons?
Is there a way to allow usage of stack by the byte or native compiler?

> You have a pretty big vector

The vector is allocated once and not on the same scale as usage,
measured by profiling or by resident memory in top.

If I let Emacs run with no GC, it will consume about 10GB before it
starts swapping and becomes unresponsive until I kill it.

> and a lot of nested iteration

I would have expected this to consume only stack memory, but it seems
that neither native compiled nor bytecode can use memory in a
stack-like way?  I'm not sure what the Elisp manual on stack allocated
objects means for native or byte compiled functions.

> cl-psetq conses.

After removing this for a combination of setq's and then removing two
unnecessary `cl-dotimes` expressions that used no CL features, I've
seen a second-run reduction down to about 8-9s.

The first run is quite a bit longer.  Although both runs GC 4-5 times,
the first run of an Emacs session is about 35s versus 8s for a second
run.  `gc-elapsed' after the first run goes from near zero to 33.7s.

The result is still on the same order of magnitude in terms of memory
usage.  The only behavior consistent with earlier memory profiling I
did is if every single floating point value ever calculated lives on a
unique piece of heap until GC'd.  It as if heap is consumed in a
straight line and there is no stack memory.

Perhaps concerning is that Emacs continues to consume the same
resident memory, about 2.1GB, indefinitely after the first run.  Is
this simply expected from a sprinkling of objects that Emacs cannot GC
and therefore never give back memory?

Is this unique to floating points?  Do I still have latent consing in
this version?  Thanks in advance.

    ;;; mandelbrot.el --- Mandelbrot -*- lexical-binding: t; -*-

    ;;; Commentary:

    ;; Render's Mandelbrot set.

    ;;; Code:

    (eval-when-compile (require 'cl-lib))

    ;;;###autoload
    (defun mandelbrot ()
      "Render a mandelbrot."
      (declare (speed 3))
      (interactive)
      (pop-to-buffer (get-buffer-create "*mandelbrot*"))
      (let ((gc-cons-threshold (* 800000 2048))
            (gc-cons-percentage 1.0))
        (let* ((cx) (cy) (zr) (zi) (v) (out) (zrt)
               (inhibit-modification-hooks t)
               (w 1600) (h 1200) (d 256)
               (output (make-vector (* w h 3) 0))
               (idx 0)
               (x0 -2.5) (y0 -1.5) (dx 4.0) (dy 3.0)
               (dxp (/ dx w)) (dyp (/ dy h)))
          (fundamental-mode) (erase-buffer)
          (set-buffer-multibyte nil)
          (insert (format "P6\n%d %d\n255\n" w h))
          (dotimes (y h)
            (dotimes (x w)
              (setf cx (+ x0 (* dxp x)))
              (setf cy (+ y0 (* dyp y)))
              (setq zr 0)
              (setq zi 0)
              (setq v (cl-dotimes (v d d)
                        (if (> (+ (* zr zr) (* zi zi)) 4) (cl-return v)
                          (setq zrt (+ (* zr zr) (- (* zi zi)) cx))
                          (setq zi (+ (* (* zr zi) 2) cy))
                          (setq zr zrt))))
              ;; samples that hit 256 wrap in the graphic to display as zero
              ;; exponential 0.8 enhances contrast a bit
              (setq out (floor (* 256 (expt (/ (float v) d) 0.8))))
              (aset output idx out)
              (aset output (+ idx 1) out)
              (aset output (+ idx 2) out)
              (setq idx (+ idx 3))))
          (insert (concat output))
          (image-mode))))

    (provide 'mandelbrot)
    ;;; mandelbrot.el ends here.