Re: Generalized-Arrays egg v1.0.1

[John Cowan]

I now understand that chicken's srfi-143 egg has a  different spec from actual SRFI-143; in particular, the egg's procedures are variadic, whereas the SRFI's procedures were intended not  to be.  Unfortunately, the SRFI specifies =, <, etc. to be variadic instead of dyadic as a result of cut and paste disease, although the SRFI sample implementations conform to my original intention.  I'm going to get the SRFI fixed if possible, or if not to issue a post-finalization recommendation (i.e. a recommendation by the SRFI author (me) that only two arguments be supported).

On Mon, Feb 5, 2024 at 8:43 PM Jeremy Steward <jeremy@thatgeoguy.ca> wrote:

On 2/4/24 05:11, felix.winkelmann@bevuta.com wrote:
>
>> 1. The egg itself is not compiled with -O3, whereas if I link to
>> (chicken fixnum) I believe that these procedures will be inlined by the
>> CHICKEN compiler when the arrays egg is compiled with -O3 or higher.
>
> That's correct. As -O3 implies "unsafe", the compiler can just ignore
> type checks and always inline numeric primitives.
>

Awesome, glad my intuition about this wasn't unfounded. This is probably
the biggest factor.

>>
>> 2. There may be some rewriting rules that the compiler uses for
>> procedures in the (chicken ...) namespace that optimizes these directly
>> into their equivalent optimized C procedures. I'm not sure the compiler
>> has the same visibility if you re-export these from behind a module, and
>> especially not if you link dynamically (any hope of inlining those is
>> thus gone forever).
>
> As a general rule all renamings and rexports preserve the run-time
> semantics - as long as a procedure definition is not wrapped inside
> another procedure, the original procedure gets compiled and any
> optimizations and inlinings the compiler performs for "known" primitives will be
> applied. Syntax- and module-specific expansions perform renaming
> only and never introduce run-time code that wasn't explicitly given by
> the user.
>

Aha, so as long as something is wrapped we don't see the same kind of
inlining.

>>
>> In fact on the latter point above, this raises an interesting question I
>> had for the mailing list: if I re-export a CHICKEN-specific procedure,
>> does the way the compiler handles translation units prevent certain
>> optimizations from applying? I've somewhat noticed that to be the case
>> but I haven't seen any writing about it nor have I understood the extent
>> to which that might affect certain SRFIs or eggs.
>
> See above - unless something is wrapped, you get the inlining. But as
> I see in the SRFI-143 spec, their operations take any number of
> arguments, where the ones from (chicken fixnum) have arity 2,
> so I assume the srfi code wraps the primitives into multi-argument
> procedures and thus preserves imports from the srfi-143 module to be inlined.
>

And yup, that's probably it. From what I can tell from the source
locally they are wrapped e.g.:

     (define (fx+ . args)
       (foldr chicken:fx+ 0 args))

>>
>>> If you have time/energy, it would be useful to make Chicken's
>>> implementation do what it was originally intended.  See the
>>> "Implementation" section in the SRFI, or feel free to ask me for
>>> explanations of details.
>
> One approach would be to extend those exports from (chicken fixnum)
> that match the multi-argument operations from srfi-143 and make the
> compiler aware of these, internally expanding the multi-arg cases into
> nested applications of inline calls to the 2-argument C runtime primitive
> ops.
>

So what would this look like? I can see fx+ above, which uses
`chicken:fx+` (prefixed, 2-arity) and foldr to accomplish its multi-arg
case. Is adding `(inline-file)` to the srfi-143.egg enough for this? I
suspect partly that the foldr and `(fx+ . args)` form in general is
going to be hard to optimize out, but perhaps a case-lambda is faster?

I suppose in pretty much all cases since I'm using the 2-arity versions
in the generalized-arrays library there's very little pushing me to use
the SRFI-143 versions, but any improvement to SRFI-143 is probably
worthwhile considering the kinds of performance differences I was
measuring with chicken-profile.

>
>> It might also be a good idea to enforce SRFI-143's behaviour around
>> overflow / underflow as part of CHICKEN's default behaviour in CHICKEN
>> 6, but I don't really know how I would get started having that discussion.
>
> The SRFI-page doesn't seem to specify any behaviour. The runtime system
> is compiled with "-fwrapv" in the hope to preserve wrapping semantics for
> integer arithmetic, but I'm not sure how reliable that is.
>

Interesting! I believed the SRFI-143 was specifying twos-complement
behaviour, but I now see that I was wrong. I would think in general that
if -fwrapv is passed, we should assume that twos-complement is used.
Recent C standards have codified this explicitly, but pretty much every
major compiler I've used over the last 10 years has all-but-standardized
twos-complement as well.

I guess as long as the runtime system is always using -fwrapv it isn't a
concern. It would be undefined behaviour without that flag though, which
I suppose someone could maybe force, with great effort?

Either way, probably not worth worrying about.

Thanks for the really informed reply Felix, this was very helpful in
understanding what's happening with SRFI-143. I can hopefully start
crafting a patch for this now.

Cheers,
--
Jeremy Steward