[Fle3-dev] Fle3 performance measurements

[Tarmo Toikkanen]

OK, I've run some preliminary performance tests with Hammerhead. I 
targeted our church server (which is something like 1.5GHz Athlon, 
1.5GB memory, 100MB NIC), which had some other users at the time, but 
nothing severe.

On a 10 minute test cycle with 100 simultaneous users and image loading 
turned on, I got the following results:

Total Requests Served        : 51406
Total Turnaround Time (msec) : 54822976
Average Request Time  (msec) : 1066
Total Responses              : 51406
Average Responses / sec      : 82.912903
Total Response Time (msec)   : 52505871
Average Response Time (msec) : 1021
Scenario Throughput          : 1796
Sequence Throughput          : 683
Run Time (sec)               : 620
Scenarios / sec              : 2.896774
Sequences / sec              : 1.101613
Content Length bytes         : 25069236
Content Length bytes / sec   : 40434
Read Length bytes            : 25069236
Read Length bytes / sec      : 40434

With image loading turned off (to simulate the fact that most browsers 
cache images quite strongly and effectively load them only one):

Total Requests Served        : 5318
Total Turnaround Time (msec) : 57047252
Average Request Time  (msec) : 10727
Total Responses              : 5318
Average Responses / sec      : 8.577419
Total Response Time (msec)   : 56509084
Average Response Time (msec) : 10626
Scenario Throughput          : 5459
Sequence Throughput          : 1898
Run Time (sec)               : 620
Scenarios / sec              : 8.804839
Sequences / sec              : 3.061290
Content Length bytes         : 74827784
Content Length bytes / sec   : 120689
Read Length bytes            : 74827784
Read Length bytes / sec      : 120689

The key values to look for here are the "Scenarios/sec" and "average 
request time". The first value tells us how many actual page requests 
the server can serve in a second, which seems to vary from 2 to 9 
requests / second, depending on image loading.

The second value, "average request time", shows how many milliseconds 
each request takes from the user point of view (time from request sent 
to response received), which ranges from 1 to 10 seconds. Obviously, 
image loading speeds this up, since serving images is a lot faster. 
Presumably the actual pages take about as long to load on both cases.

This estimate of 2-9 requests/second has to be considered optimistic, 
since the test scenario doesn't cover the entire database and all 
pages, but concentrates on webtops, course pages and notes, and just a 
few of those as well. I've noticed that Zope's caching plays a big role 
here and really speeds up subsequent requests for the same object.

Anyway, if we assume that each page contains about 10 seconds worth of 
material (on average), then we can support around 20-90 simultaneous 
users. This 10 seconds probably is a low estimate, so the figure might 
be higher, but anyway, currently we're talking about figures of this 
magnitude.

And yes, this is on a *nix machine. Apparently the performance in 
Windows + IIS is _really_ horrible, or so I've heard.

--
Tarmo Toikkanen
http://www.iki.fi/tarmo/
The Fle3 Team
http://fle3.uiah.fi