[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Are frozen files really that hot?
From: |
Daniel Goldman |
Subject: |
Are frozen files really that hot? |
Date: |
Tue, 01 Jul 2014 14:25:00 -0700 |
User-agent: |
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 |
Hi,
The manual says: "Later calls, containing the -R option, are able to
reload the internal state of m4, from base.m4f, prior to reading any
other input files. This means instead of starting with a virgin copy of
m4, input will be read after having effectively recovered the effect of
a prior run. In our example, the effect is the same as if file base.m4
has been read anew. However, this effect is achieved a lot faster."
"A lot faster" sounds promising. But also kind of vague. 100% faster?
200% faster? And of course the frozen file just affects the loading
phase. If the loading phase is just a fraction of the processing cycle,
it might not matter much.
Anyway, in the course of doing some development, I figured I would do a
benchmark on frozen file performance. The task is creating some 350 web
pages using m4. There are about 450 macros. For the benchmark, I limited
the test to just the m4 part, customized the shell scripts not to do the
other parts (sed, grep, etc). On a single user linux e3-1245 v3, with
crontab turned off, I either 1) make the frozen file first, and use it
repeatedly to make the 350 outputs, or 2) use the "unfrozen" file each
time, the so-called "virgin copy". :) I used /usr/bin/time to measure
the time. Here are the results:
**************************
freeze=y # use m4 frozen file? 1.65 seconds
freeze=n # use m4 frozen file? 1.80 seconds
freeze=y # use m4 frozen file? 1.22 seconds
freeze=n # use m4 frozen file? 1.98 seconds
freeze=y # use m4 frozen file? 1.04 seconds
freeze=n # use m4 frozen file? 1.80 seconds
freeze=y # use m4 frozen file? 1.27 seconds
freeze=n # use m4 frozen file? 1.96 seconds
freeze=y # use m4 frozen file? 1.36 seconds
freeze=n # use m4 frozen file? 1.88 seconds
{totals[y]} = 6.54
{totals[n]} = 9.42
**********************
That's a 31% improvement. (9.42 - 6.54) / 9.42 = .306
Or one could say the non-frozen test takes 44% longer.
That's better than nothing, but I was hoping for better. I realize this
is just one test. And sorry not feasible for me to provide the details,
so a synthetic benchmark would also be helpful. Are there other
real-world (or synthetic) benchmarks for frozen files, or any cases
where frozen files are really that helpful for some task?
**********************
Generally, I look for much greater speed up than 31%, especially as the
overall speed-up (including the non-m4 tasks) would be even lower. For
example, I recently got into multi-thread programming in C and using
shell scripts. It was kind of a bother setting up, but the speed up for
various tasks was dramatic. For the same task making the 350 web pages
(with all steps included, which take up much more time than the m4 step)
the single thread time was 15.5 seconds, the multi-thread time (using
xjobs) was 4.3 seconds. And only a fraction of that time is taken by m4,
which is already pretty fast.
Also, setting up scripts to use frozen files requires an additional
step, a small negative (it's not that difficult).
If there were no downside to frozen files, I wouldn't bother. One could
take them or leave them. I am in favor of giving users choice.
But I imagine there is a fair amount of infrastructure related to frozen
files, with programming, source code, file formats, systems analysis,
etc. It would seem to make the software more complex. My guess is that
frozen files introduce some complications.
If only minimal effort is required related to frozen files (hard for me
to believe), then I guess there is no problem. Otherwise, I am wondering
if frozen files are worth it. At this point, based on my benchmarks, I
don't plan to use frozen files.
I am wondering if maybe frozen files are a failed experiment, or perhaps
outmoded by improvements in cpu performance. Maybe frozen files were
useful decades ago. But I would suggest that maybe future effort related
to frozen files might better be spent improving and fixing other
somewhat obvious things in m4. I would even suggest maybe frozen files
might be deprecated at some point. Maybe "less is more" in this case.
If there are other benchmarks showing frozen files really are hot, I
take all this back. It's always possible I did something wrong. But the
whole point of frozen files is to speed things up, yet doing a keyword
search for "benchmark" on this group, this seems to be the first post
that actually did a benchmark (of any kind). So I think this is worth
bringing up.
Thanks,
Daniel
- Are frozen files really that hot?,
Daniel Goldman <=