The advantages of this benchmark are:
- it is a multiclient benchmark
- the client side implementation does not influence benchmark results
which is the way it should be for a server benchmark
The disadvantages of this benchmark are:
- the current workload is not realistic
I need help to:
- Review benchmark methodology/implementation
- Create a realistic workload (or maybe different workloads for
different environments)
LADDIS was originally developed by six vendors who saw the need for
better NFS benchmarks (Legato, Auspex, DEC, Data General, Interphase,
and Sun). The original work group then took the benchmark to SPEC for
further development and promotion as an industry standard. LADDIS has
considerably helped performance evaluation of NFS servers, thereby
contributing to the development of better servers.
I want to make the same thing happen for HTTP servers. It's important
that such work not be done by any single vendor, nor with any single
narrow viewpoint of the requirements. Would people in this group be
interested in working together to create an industry standard benchmark
for HTTP servers?
I would like to thank the SMCC Performance Engineering Group for their
support in this effort.
Regards,
Prasad
Note: this table illustrates the type of output generated by this
benchmark. The actual numbers aren't meaningful due to the dummy
workload and uncontrolled test environment.
Benchmark Results
************************************************************************
Aggregate Test Parameters:
Number of processes = 1
Requested Load (HTTP operations/second) = 10
Warm-up time (seconds) = 1
Run time (seconds) = 120
Aggregate Results for 1 Client(s), Tue Feb 28 13:37:34 1995
HTTP Server Benchmark Version 1, Creation - 15 February 1995
--------------------------------------------------------------------------
HTTP Target Actual HTTP HTTP Mean Std Dev Std Error Pcnt
Op HTTP HTTP Op Op Response Response of Mean,95% of
Type Mix Mix Success Error Time Time Confidence Total
Pcnt Pcnt Count Count Msec/Op Msec/Op +- Msec/Op Time
--------------------------------------------------------------------------
get 80% 82.0% 32 0 3271.62 822.29 9.93 85.9%
head 10% 5.1% 2 0 2450.00 28.69 7.42 4.0%
post 10% 12.8% 5 0 2437.80 43.81 5.80 10.0%
put 0% 0.0% 0 0 0.00 0.00 0.00 0.0%
delete 0% 0.0% 0 0 0.00 0.00 0.00 0.0%
checkout 0% 0.0% 0 0 0.00 0.00 0.00 0.0%
checkin 0% 0.0% 0 0 0.00 0.00 0.00 0.0%
showmethod 0% 0.0% 0 0 0.00 0.00 0.00 0.0%
link 0% 0.0% 0 0 0.00 0.00 0.00 0.0%
unlink 0% 0.0% 0 0 0.00 0.00 0.00 0.0%
--------------------------------------------------------------------------
INVALID RUN reported for Client 1 (haven).
--------------------------------------------------------
| AGGREGATE RESULTS SUMMARY |
--------------------------------------------------------
HTTP THROUGHPUT: 0 Ops/Sec AVG. RESPONSE TIME: 3122.6 Msec/Op
HTTP MIXFILE: [ default ]
AGGREGATE REQUESTED LOAD: 10 Ops/Sec
TOTAL HTTP OPERATIONS: 39 TEST TIME: 119 Sec
NUMBER OF CLIENTS: 1
------------------------------------------------------------------------
************************************************************************