Latest benchmarks

[John L Daschbach]

I tried the latest benchmark.m file and found it had some bugs plus it
does not give close to 'real' results for my two machines.  

First bug:  The following code runs the cputime() call on a machine
without cputime().  (P/120 OS/2).  It appears that functions are
defined at the top level and the second definition overwrites the
first.  I put printf() statements in various places and it's true that
the disp("Warning") is called when you don't have cputime().  But when
running the call actually goes to the second one.  It appears the
parser simply reads all function definitions, ignoring the if()
statement.  

if (cputime() == 0)
  # Use these function if cputime() does not work on this particular
  # port of octave.  In this case, time will be computed on a wall
  # clock, and will make sense only on a machine where no other 
  # processes are consuming significant cpu time while the benchmark
  # is running.
  disp ...
    ("WARNING: if other processes are running the figures will be inaccurate");
  function t = bm_st ()
    t  = clock();
  endfunction
  function et = bm_et (t);
    et = etime(clock(),t);
  endfunction
else
  function t = bm_st ()
    t = cputime();
  endfunction
  function et = bm_et (t)
    et = cputime() - t;
  endfunction
endif


Second bug:  The 'global s0, t' produces an error for me with Octave
1.1.1 on both OS/2 and AIX.  Removing the comma fixes this.

Third problem.  The results make the Pentium seem much faster.  In
part this may be because of the small matrix size used and the calling
overhead of the script.  On an RS6000 these small matrices take so
little time that a large part of the time is due to the octave
language calls to things like the timing routine.  In other words this
benchmark code is no good.

For instance, the benchmark gives matrix inversion timings which are
between 2 and 3 times faster on the AIX box.  However when I run
simple tests using tic() and toc() or clock() and etime() I find that
the AIX box is between 5 and 6 times faster for 256x256 and 512x512.
The AIX is used by other people and the OS/2 is single user with
nothing but networking running.  One would expect that actual cpu
timings on the AIX box would be faster than the tic() or toc() timings
but they can't be any slower.  

The LSODE result is interesting.  I have found lsode() in octave very
slow on the AIX box.  What is interesting is that we find the RS6000
boxes to be very good all around performers for numerical stuff.  We
have had boxes from most everyone and RS6000's are the favorites for
small machines (<1G core).  The main application is electronic
structure codes, (DF and MCSCF/CI).  For this the RS6000 is fast and
stable.  Machines are benchmarked using the codes that are actually
run, and for electronic structure this includes doing IO and running
with other jobs.

Here are my results:

Pentium 120 

[E:\home]octave
Octave 1.1.1 for OS/2 2.x and Warp.
(Patchlevel 1.1.1-b02).
Copyright (C) 1992, 1993, 1994, 1995 John W. Eaton.
OS/2-Port by Klaus Gebhardt, 1994 - 1995.
This is free software with ABSOLUTELY NO WARRANTY.
For details, type `warranty'.

[home]:(1)>benchmark
Octave benchmark version 0.01
WARNING: if other processes are running the figures will be inaccurate
Using CLOCK Timings
Speed of octave 1.1.1 on i486 running OS/2 relative to a Sun Sparc 2
test #1         3.11 +/- 21.3% (178 runs)
test #2         3.60 +/- 18.5% (51 runs)
test #3         6.17 +/- 2.3% (10 runs)
[home]:(2)>

RS6000/590

Speed of octave 1.1.1 on rs6000-ibm-aix3.2.5 relative to a Sun Sparc 2
[d3h486]:(1)>quit
lurch(45)% octave
Octave, version 1.1.1.
Copyright (C) 1992, 1993, 1994, 1995 John W. Eaton.
This is free software with ABSOLUTELY NO WARRANTY.
For details, type `warranty'.

[d3h486]:(1)>benchmark
Octave benchmark version 0.01
Speed of octave 1.1.1 on rs6000-ibm-aix3.2.5 relative to a Sun Sparc 2
test #1         6.73 +/- 0.0% (10 runs)
test #2         10.53 +/- 0.0% (10 runs)
test #3         2.76 +/- 0.9% (10 runs)
[d3h486]:(2)>