Resource Troubleshooting

"The server has gotten slow" is a complaint that can trace back to four entirely different causes: not enough CPU, RAM running out, disk I/O saturation, or a full disk. Process Monitoring and Process Memory already covered CPU and memory in depth at the level of a single process, and Disk and File System Basics covered disk capacity. But when a problem actually shows up, the first question is different: "which of these four is it?" This article is about exactly that first, fast triage step — comparing CPU, memory, and disk with one unified view, then moving into a deep dive on whichever resource turns out to be the culprit.

Why One Tool Isn't Enough

top shows CPU and memory together, but tells you almost nothing about disk I/O. free only covers memory. df only covers disk capacity. If the actual problem is disk speed and you only check top, CPU and memory can look completely normal — and time gets wasted looking in the wrong place. That's why the first triage step should always be a tool that shows all resources at once.

vmstat: Four Resources on One Screen

vmstat 2 5

This prints statistics every 2 seconds, 5 times total:

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0      0 3400200 812400 2200800    0    0    12    30  140  310  8  3 88  1  0
 6  4      0 3398100 812400 2201900    0    0  1840  2200  310  520 10  6 40 44  0

The columns fall into four groups:

The second row paints a completely different picture from the first: both r (6) and b (4) are elevated, and wa is 44 — this means the CPU itself is not busy (us+sy = 16), but is instead sitting idle waiting for the disk to respond.

Load Average: the First Number, and What It Hides

Before vmstat, the fastest single glance is the load average — the three numbers uptime (and top) show for the last 1, 5, and 15 minutes:

uptime

 12:40:02 up 3 days,  4:11,  2 users,  load average: 7.82, 4.10, 2.05

Two things make this number widely misread:

• It is relative to core count. A load of 7.82 is saturation on a 4-core box but comfortable on a 16-core one — always divide by the output of nproc. The rising trend here (7.82 over 1 min vs 2.05 over 15 min) says the pressure is recent and growing, which is often more useful than the absolute value.
• On Linux it counts more than CPU. Unlike most other Unixes, Linux load includes processes in uninterruptible sleep (D state, waiting on I/O), not just those waiting for CPU. That is why a machine with idle CPUs can still show a load of 8 — the same disk-bound situation the b and wa columns of vmstat reveal. A high load with low us/sy is the signature of an I/O problem, not a CPU one.

In short, load average tells you that the system is under pressure and how it is trending; vmstat is still what tells you which resource is responsible.

Which Resource to Check First: A Decision Table

A single vmstat snapshot, combined with the table below, helps choose the next step:

iostat: Which Disk Is at Fault

vmstat only gives the overall I/O picture — it doesn't say which device is the problem. For that, iostat, from the sysstat package, is used:

sudo apt install sysstat

sudo is needed here because the new package is installed into system directories — the same installation process covered in apt and Repositories.

iostat -x 2 3

Device            r/s     w/s   rkB/s   wkB/s  await  %util
nvme0n1           4.50   82.00   180.0  9840.0  38.20  96.40

The important columns:

• %util — the percentage of time the device was busy. A value near 90–100% means the disk is nearly saturated, and new requests are queuing up;
• await — the average time a request waits, in milliseconds. For a modern SSD/NVMe drive, an await above a few milliseconds is already suspicious, and tens of milliseconds indicates a serious problem.

In the output above, %util is 96% and await is 38ms — confirming exactly what the high wa in vmstat suggested: the nvme0n1 device is saturated.

Disk Space: A Separate, but Often Confused, Problem

Disk I/O (speed) and disk space (capacity) are two independent problems. A disk can be very fast but completely full, or the reverse. A quick check:

df -h /

Full analysis — inode exhaustion, "the disk is full but du shows little usage," and other tricky cases — is already covered in depth in Disk and File System Basics; at the triage stage, a quick "full or not" answer from df -h is enough.

Practical Scenario: "The Site Is Responding Slowly"

Monitoring reports that web server responses have gotten slower. Every command below is read-only.

Step 1. Get the overall picture.

vmstat 2 5

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 3  5      0 2100400 402100 3800200    0    0   210  8400  380  610 12  8 15 65  0

wa is 65, b is 5. The CPU itself (us+sy = 20) isn't particularly busy, but a lot of processes are waiting on disk responses.

Step 2. Identify which device is bottlenecked.

iostat -x 2 3

Device            r/s     w/s   rkB/s   wkB/s  await  %util
nvme0n1          12.00  340.00   480.0 42000.0  61.80  98.10

%util is nearly 100%, await is 62ms — the disk is saturated.

Step 3. Find which process is doing this writing.

ps -eo pid,stat,comm | grep ' D '

A D in the STAT column means uninterruptible sleep — the process is specifically waiting on a disk I/O response. The full explanation of ps state codes is in Process Monitoring.

7801 D    postgres

Conclusion: the slowdown isn't CPU or RAM — it's heavy disk writing generated by the postgres process. The next step is finding out why this process is writing so much (a large indexing operation, or a poorly written query, for example), using the application's own log source and the techniques covered in Log Analysis.

Common Mistakes

Checking Only top and Skipping Disk I/O

top can show a high %wa value, but it doesn't say which device or which process is causing it. In that situation, iostat identifies the device and the D state in ps finds waiting processes; to attribute I/O to a specific process directly, sudo iotop -o (interactive, shows only processes actually doing I/O) or pidstat -d 2 (from the same sysstat package, script-friendly) are the purpose-built tools.

Confusing %util with CPU Percentage

%util in iostat is the percentage of time the disk device was busy, not the processor. 100% CPU and 100% disk %util are completely different problems that call for different fixes.

Treating Disk Space and Disk Speed as the Same Problem

df -h might report "80% used," but that has nothing to do with I/O slowness. They are independent measurements — one is capacity, the other is throughput.

Exercises

1. Run vmstat 2 5 on your own system, write down the values in the r, b, and wa columns, and justify whether they look normal or problematic.
2. Install iostat with sudo apt install sysstat and interpret the %util and await columns from iostat -x 2 3 for your own disk.
3. On a test VM, generate artificial disk load with dd if=/dev/zero of=/tmp/testfile bs=1M count=2000 (disposable VM only, inside /tmp), watch wa change with vmstat 1 5 while it runs, then remove the test file: rm /tmp/testfile.
4. Pick three scenarios from the decision table above and write out, from memory, the sequence of checks you would run for each.

Verification criterion: for exercise 3, you should be able to point to the specific row in vmstat output where wa rises in response to the dd command, and explain why us/sy stay comparatively low during the same window.

References

• man7.org: vmstat(8): https://man7.org/linux/man-pages/man8/vmstat.8.html
• man7.org: iostat(1): https://man7.org/linux/man-pages/man1/iostat.1.html
• Sysstat Documentation: https://github.com/sysstat/sysstat
• man7.org: ps(1) — process state codes: https://man7.org/linux/man-pages/man1/ps.1.html