iostat, vmstat, netstat – Performance Monitoring & Tuning in Unix & Linux

iostat, vmstat, netstat are the tools in Linux and Unix operating system which provides important system performance data relating to disk, memory network and cpu. By analyzing the data it is possible to determine the potential bottleneck which is impacting the system performance and take corrective action accordingly.

iostat , vmstat and netstat are three most commonly used tools for performance monitoring . These comes built in with the operating system and are easy to use . iostat stands for input output statistics and reports statistics for i/o devices such as disk drive. vmstat gives the statistics for virtual Memory and netstat gives the network statistics .

Following pages describes these tools and their usage for performance monitoring explains their syntax , examples and explanation of results and solution for the common problems

This examples are with reference to Solaris but these tools are available in Linux and Other Unix distributions with slight difference in syntax and output.

iostat – Input Output statistics

iostat reports terminal and disk I/O activity and CPU utilization. The first line of output is for the time period since boot & each subsequent line is for the prior interval . Kernel maintains a number of counters to keep track of the values.

iostat’s activity class options default to tdc (terminal, disk, and CPU). If any other option/s are specified, this default is completely overridden i.e. iostat -d will report only statistics about the disks.

iostat syntax

Basic synctax is iostat interval count
option – let you specify the device for which information is needed like disk , cpu or terminal. (-d , -c , -t or -tdc ) . x options gives the extended statistics .

interval – is time period in seconds between two samples . iostat 4 will give data at each 4 seconds interval.

count – is the number of times the data is needed . iostat 4 5 will give data at 4 seconds interval 5 times

iostat Example

$ iostat -xtc 5 2
                          extended disk statistics       tty         cpu
     disk r/s  w/s Kr/s Kw/s wait actv svc_t  %w  %b  tin tout us sy wt id
     sd0   2.6 3.0 20.7 22.7 0.1  0.2  59.2   6   19   0   84  3  85 11 0
     sd1   4.2 1.0 33.5  8.0 0.0  0.2  47.2   2   23
     sd2   0.0 0.0  0.0  0.0 0.0  0.0   0.0   0    0
     sd3  10.2 1.6 51.4 12.8 0.1  0.3  31.2   3   31

The fields have the following meanings:
      disk    name of the disk
      r/s     reads per second
      w/s     writes per second
      Kr/s    kilobytes read per second
      Kw/s    kilobytes written per second
      wait    average number of transactions waiting for service (Q length)
      actv    average number of transactions  actively being serviced 
(removed  from  the  queue but not yet completed)
      %w      percent of time there are transactions  waiting
              for service (queue non-empty)
      %b      percent of time the disk is busy  (transactions
                  in progress)

iostat Results and Solutions

The values to look from the iostat output are:
* Reads/writes per second (r/s , w/s)
* Percentage busy (%b)
* Service time (svc_t)

If a disk shows consistently high reads/writes along with , the percentage busy (%b) of the disks is greater than 5 percent, and the average service time (svc_t) is greater than 30 milliseconds, then one of the following action needs to be taken

1. Tune the application to use disk i/o more efficiently by modifying the disk queries and using available cache facilities of application servers .
2. Spread the file system of the disk on to two or more disk using disk striping feature of volume manager /disksuite etc.
3. Increase the system parameter values for inode cache , ufs_ninode , which is Number of inodes to be held in memory. Inodes are cached globally (for UFS), not on a per-file system basis
4. Move the file system to another faster disk /controller or replace existing disk/controller to a faster one.

Next Page – vmstat