On Fri, Sep 26, 2008 at 5:32 PM, Marc Feeley
<address@hidden> wrote:
The conclusion from my benchmarks is quite different. Chicken does
not
outperform Gambit on these benchmarks. There is so little other
stuff
happening than call/cc in these benchmarks that it would appear
that the
performance of call/cc in Chicken and Gambit is essentially the
same (to
within a few percent).
Why not simply say: chicken and gambit are roughly in the same
ballpark?
In the end, I have learned that nearly every performance assumption
I made
was wrong, and I'm a pretty experienced Scheme coder. Performing
benchmarks
like this and trying to extract any kind of practical relevance from
the fact that
program X on implementation Y with optimization settings Z takes 2%
longer than
on implentation Q. Are you sure you have built both implementations
with
maximal performance settings? Have you measured how much runtime-
performance
the memory patterns in this particular benchmark have caused? How do
you know
how your system configuration and hardware setup influences the
outcome?
Have you used optimal optimization settings for all implementations
for this
benchmark? Have you analyzed the compiler output to look for
opportunities
to tweak those settings for this particular benchmark? Do you know
enough
about chicken's internals and compiler options to chose the optimal
combination
(you couldn't, just as I couldn't for Gambit). It's all just
assumptions.
The very reason Scheme and Lisp have so little acceptance and are
not more
widespread is that its implementors are so obsessed with performance
(for
hystorical raisins, of course), instead of making their implementation
easier to work
with, more practical and more useful.
Nevertheless I understand this obsession, its lots of fun, after
all. :-)
So: CheneyOnTheMTA is an elegant concept that unifies fast first-class
continuations, fast allocation, generational GC and not-too-
difficult FFI
in a relatively simple framework. Chicken's compiler is sufficient,
but there are
many opportunities to improve performance, some of which will be
addressed, but which aren't really that important. A real module
system
(soon to come!) and 400+ libraries is what will make users happy,
not 5% better
performance.
I believe that CheneyOnTheMTA is more memory-efficient than other
Lisp-implementation techniques. I also believe that the CPS-output of
this scheme
is more C-compiler friendly and easier to compile on stock machines.
I believe
that COTMTA (that's a nice abbreviation - I think I'll use that from
now on) makes
cross-module calls more efficient than trampoline-style, which is
important
for large code-bases that use separate compilation and dynamically
loaded
plugins. These are all assumption that may possibly be completely
wrong.
Keep up the good work, Marc! Gambit is cool. But chicken is
better. ;-)
cheers,
felix