http://lbs.sourceforge.net/
Linux Benchmark Suite Homepage
Mission
To provide a wide range of both synthetic and application benchmarks for Linux as well as information and tools to tune and run them.
Introduction
This project is made up of various parties interested in measuring aspects of linux performance. Some members are interested in performance of a specific workload on normal PC architectures, some are comparing specific hardware capabilities,some are trying to push linux's scalibility limits on larger systems, and some are trying to measure performance of specific applications.
One of the problems with benchmarking on linux is the ability to talk about benchmark numbers in the open as certain benchmarks have specific restrictions on their use, usually requiring the results to be audited and approved. We hope to develop open benchmarks to use in substitution of restricted benchmarks.
Project Resources
Project management tools and mailing lists can be found on the SourceForge site.
Benchmark Tools
Quick-Hit Benchmarks
"Quick-hit" benchmarks are simple tests to measure a certain aspect of performance, but usually do not give a larger perceptive of system performance.
- Ttcp - measures the point-to-point bandwidth over a network connection
- Ping - can measure the latency of a network connection
- Hdparm - "-t" and "-T" options can be used to measure disk-to-memory (disk reads) transfer rates
- Dga - the "-b" option measures CPU/video memory bandwidth
Synthetic Benchmarks
Synthetic benchmarks are similar to "quick-hit" tests in that they are not meant to represent a "real-world" workload, but can often be useful in measuring the maximum capacity of specific aspects of a system.
- lmbench - a GPL'd suite of atomic benchmarks, no publishing restrictions
- UnixBench - a fundamental high-level Linux benchmark suite, Unixbench integrates CPU and file I/O tests, as well as system behaviour under various user loads
- AIM9 - the AIM Independent Resource Benchmark exercises and times each component of a UNIX computer system, independently. The benchmark uses 58 subtests to generate absolute processing rates, in operations per second, for subsystems, I/O transfers, function calls, and UNIX system calls. GPL'd and can be published under the "non-audited" clause.
- Netperf - a sophisticated network and filesystem benchmark, freely available, publishable?
- SSLperf - open source web benchmark designed to measure performance of SSL operations
- dbench - similar workload to netbench but GPL'd and much easier to config and run (doesn't require clients), suite also includes tbench and smbtorture, no publishing restrictions
- Bonnie - io throughput benchmark, GPL'd with no publishing restrictions
- Bonnie++ an enhanced version of bonnie written in C++, GPL'd with no publishing restrictions
- Iozone is useful for performing a broad filesystem analysis of a vendor's computer platform. The benchmark tests file I/O performance for the following operations: Read, write, re-read, re-write, read backwards, read strided, fread, fwrite, random read, pread ,mmap, aio_read, aio_write. It has recently added tests for NFS, CIFS, and distributed/cluster systems.
- SPEC CPU2000 - benchmark suite designed to evaluate raw cpu and compiler power
- BYTEmark - CPU benchmark suite, reporting CPU/cache/memory, integer and floating-point performance
- Cachebench - measures bandwidth of the memory subsystem (L1, L2 and main memory)
- Stream - measures sustainable memory bandwidth vs. FPU performance
- SPECviewperf - synthetic graphics benchmark
- Xengine - a little X window toy that shows the speed with which a system will redraw a coloured bitmap on screen (a simulation of a four cycle engine), availible from www.tux.org/pub/benchmarks/
- Xbench Xserver benchmark
- XMark93 - Xserver benchmark (part of SPEC, defunct?)
Application Benchmarks
- Open Source Database Benchmark - GPL'd benchmark based on the AS3AP database benchmark with no publishing restrictions
- VolanoMark - a Java-based chat room benchmark, freely availible, publishable?
- chat - a GPL'd chat room benchmark written in C, designed to emulate VolanoMark behavior, no publishing restrictions
- Postal SMTP mail delivery test and POP mail reception test
- AIM7 - the AIM Multiuser Benchmark - Suite VII tests and measures the performance of Open System multiuser computers. It comes with preconfigured workload mixes for Multiuser/Shared System, Compute Server, Large Database,and File Server. Publishable under the "non-audited" clause.
- Webstone - web server benchmark, freely availible, publishable?
- MDBNCH - a large FORTRAN 77 application benchmark, accesses a large data set in a very irregular pattern, generates misses in both the L1 and L2 caches
- TPC-A - now defunct Online Transaction Processing (OLTP) database benchmark
- TPC-B - now defunct Online Transaction Processing (OLTP) database benchmark
- TPC-C - current industry standard Online Transaction Processing (OLTP) database benchmark
- TPC-D - now defunct Decision Support System (DSS) database benchmark
- TPC-H - current industry standard ad-hoc DSS database benchmark
- TPC-R - current industry standard Business Reporting database benchmark
- TPC-W - current industry standard E-commerce database benchmark
- SPECapc - graphics application benchmark
- SPEC HPC96 - hpc application benchmark
- SPEC JBB2000 - server-side Java benchmark
- SPEC JVM98 - client-side Java benchmark
- SPEC SFS97 - NFS benchmark
- SPEC WEB99 - Web server benchmark
- SPEC SDM - defunct multi-user system benchmark
- NAS Parallel Benchmark - Computational Fluid Dynamics benchmark, source freely available, publishable?
- ARC2D Benchmark - scientific benchmark, source freely available, publishable?
- PARKBENCH - PARellel Kernels and BENCHmarks
- High Performance Linpack Benchmark
Findings and Whitepapers
Other useful resources
Last updated: 5/15/02 Any problems, additions, etc , please send email to the Webmeister.