Linux Kernel Performance
|LTC Home | Linux Scalability Effort | LinuxKernelPerformance Project | Performance Tools | Disk I/O | IOZONE | LMBench | Netbench | Netperf | SPEC SDET | SPECWeb99 | tiobench | VolanoMark|
This page lists a number of performance tools that are used by the LTC Kernel Performance Team.
It should be noted that the gprof output from annotated call-graph mode combines raw profile data (obtained by recording the instruction address at each clock tick) with information about function entry as generated by mcount. The data about time spent in each function, thus, is not an exact measurement (since there is no recording at function exit) but an approximation based upon the data collected by the flat profiling.
Kernprof has other interesting modes as well. Among those are the some that allow profiling by programming one of the Pentium III performance counters. This can be combined with call-graph profiling (the flat profiling information is collected when the performance counter overflows rather than when the timer ticks) and, of course with flat profiling. For example, one can cause an interrupt to occur every million instructions or every 100,000 cache-line misses. Just as time-based profiling shows where the kernel spends its time, an instruction-based profile shows where the kernel executes most of its instruction and a cache-line based profile shows where the kernel takes most of its cache-line misses. These latter types of profiling can provide additional insight into kernel performance.
For more information on kernprof, visit SGI-kernprof.
For more information on Lockmeter, visit SGI-lockmeter.
For more information on gcov, visit http://www-es.fernuni-hagen.de/cgi-bin/info2html?(gcc)Gcov
Strace is a very useful tool for analyzing the system call activity on a system. A variety of options allows for detailed as well as summarized reports.
For more information on Strace, visit The Strace Homepage .
LTT collects trace from a fixed (but expandable with DProbes) instrumentation of the kernel. Post processing is by a graphical tool, which does not appear to be widely used for performance analysis. The tool lacks post processing tools for more detailed analysis of the data collected by the tracing.
For more information on LTT, visit OperSys.
Dynamic Probes is a generic and pervasive debugging facility that will operate under the most extreme software conditions such as debugging a deep rooted operating system problem in a live environment, for example in the page-manager of the kernel or perhaps a problem that will not re-create easily in either a lab or production environment. For such inaccessible problem scenarios Dynamic Probes not only offers a technique for gathering diagnostic information but has a high probability of successful outcome without the need to build custom modules for debugging purposes.
The DProbes facility can be used to insert software probes dynamically into executing code modules. When a probe is fired, a user written probe-handler is executed. The probe-handler is a program written in an assembly-like language, based on the Reverse Polish Notation. Instructions are provided to enable the probe-handler to access all the hardware registers, system data structures and memory.
Some of the unique aspects of the Dynamic Probes facility are:
last updated: 10/01/2002