RAID (mdadm)

Partitions covered splitting a single disk into parts. RAID works in the opposite direction: it combines multiple physical disks into a single logical block device with improved reliability or speed. Whether data survives a failed disk, or writing to several disks in parallel improves throughput, depends entirely on the RAID level chosen.

RAID and LVM are often framed as competitors, but they actually solve different problems and are typically used together: RAID provides reliability and speed at the disk level, while LVM provides flexible volume management on top of that RAID array. In practice, the usual stack looks like: physical disks → RAID array (mdadm) → LVM PV → VG → LV → filesystem.

What You Will Learn

By the end of this article you will be able to:

• explain the differences between RAID levels 0, 1, 5, 6, and 10, and when each one fits;
• create a software RAID array with mdadm --create;
• check array status with /proc/mdstat and mdadm --detail;
• ensure an array reassembles automatically after a reboot via mdadm.conf;
• carry out the process of removing a failed disk from a RAID array and adding a new one;
• explain why RAID is not a substitute for backups.

What RAID Is and Why It's Needed

RAID (Redundant Array of Independent Disks) combines multiple disks into a single logical device, giving one or both of these two benefits:

• Redundancy — data survives when one or more disks fail;
• Performance — throughput improves through parallel reads/writes across multiple disks.

On Linux, software RAID is created and managed with mdadm (multiple disk administration). It's cheaper and more flexible than hardware RAID controllers and works with any disk combination, but RAID computation (parity, etc.) consumes CPU resources.

RAID Levels: 0, 1, 5, 6, 10

When to choose which level:

• RAID 0 — only when speed matters and data loss is acceptable (e.g. a temporary cache or reproducible computation results). Not recommended for production data.
• RAID 1 — two disks, a simple and reliable choice, e.g. for a system disk (/).
• RAID 5 — three or more disks, when capacity efficiency matters and tolerating one disk failure is enough. On large disks (e.g. 8+ TB), rebuild time is long, raising the risk of a second disk failing during that window.
• RAID 6 — the same as RAID 5, but tolerates two disk failures; recommended over RAID 5 for larger arrays.
• RAID 10 — for workloads like databases that need both speed and reliability, the best choice if disk count (at least 4) and budget allow.

Required Package

On the Debian/Ubuntu family:

sudo apt update
sudo apt install mdadm

During installation, mdadm may ask for an email address for array monitoring — this can also be configured later in /etc/mdadm/mdadm.conf.

Before Creating an Array: Checking the Disks

Before creating a RAID array, check that the disks hold no existing data or old RAID metadata. This is a read-only command that changes nothing:

sudo mdadm --examine /dev/sdb /dev/sdc
lsblk

If an old RAID superblock is found (mdadm --examine returns a result), it may need to be cleared before creating a new array:

sudo mdadm --zero-superblock /dev/sdb

Creating a RAID 1 Array

Suppose /dev/sdb and /dev/sdc are two empty disks of the same size.

1. Create the array

sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc

• --create /dev/md0 — the new array's name (md0, md1, ... assigned sequentially);
• --level=1 — the RAID level;
• --raid-devices=2 — the number of disks in the array.

After this, a background sync (resync) begins, bringing both disks to the same state. This can take hours on large disks, but the array can be used while this is in progress.

2. Check status

cat /proc/mdstat

Sample output:

Personalities : [raid1]
md0 : active raid1 sdc[1] sdb[0]
      10476544 blocks super 1.2 [2/2] [UU]
      [=========>...........]  resync = 45.2% (4736000/10476544) finish=2.1min speed=42000K/sec

unused devices: <none>

[UU] means both disks are "up" (healthy). If a disk fails, this shows [U_] or [_U] instead.

More detail:

sudo mdadm --detail /dev/md0

This command gives a full report on the array's state, each disk's role (active, spare, faulty), and the RAID level.

3. Create a filesystem and mount it

sudo mkfs.ext4 /dev/md0
sudo mkdir -p /mnt/raid1
sudo mount /dev/md0 /mnt/raid1

LVM can also be built on top of a RAID array — for example, adding /dev/md0 as a PV with pvcreate /dev/md0 into a VG, then continuing with the ordinary LVM layers.

Making the Array Permanent: mdadm.conf

For an array created with mdadm --create to reassemble automatically after a reboot, its configuration needs to be written into mdadm.conf:

sudo mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf

This command appends the current array's UUID and composition to mdadm.conf. On Debian/Ubuntu, refreshing the initramfs afterward is also recommended, or the array might not assemble correctly at boot:

sudo update-initramfs -u

For a permanent mount, an entry is added to /etc/fstab referencing the UUID — covered in depth in fstab:

sudo blkid /dev/md0

Replacing a Failed Disk

RAID's main practical benefit shows up here: data survives a disk failure, but removing the failed disk and adding a replacement correctly still requires the right procedure.

1. Identify the failed disk

cat /proc/mdstat
sudo mdadm --detail /dev/md0

mdadm --detail marks a failed disk as faulty. If a disk isn't yet marked faulty but is confirmed to have failed (for example, disk errors show up in dmesg), it can be manually marked:

sudo mdadm --manage /dev/md0 --fail /dev/sdc

2. Remove the failed disk from the array

sudo mdadm --manage /dev/md0 --remove /dev/sdc

3. Physically replace the disk

At this point, unless hot-swap is supported, the server is powered down and the failed disk is physically replaced with a new one, which should generally be the same size or larger than the old one.

4. Add the new disk to the array

sudo mdadm --manage /dev/md0 --add /dev/sdc

Once added, mdadm automatically begins rebuilding data onto the new disk:

cat /proc/mdstat

Once the rebuild finishes, the [UU] state is restored.

Spare Disks, Scrubbing, and Monitoring

Three operational details separate a RAID setup that survives a real incident from one that only looks healthy in a demo.

Hot Spares

A spare disk sits idle in the array and is pulled in automatically the moment an active disk is marked faulty — the rebuild starts without waiting for a human. Add one at creation time or later:

sudo mdadm --create /dev/md0 --level=5 --raid-devices=3 --spare-devices=1 \
  /dev/sdb /dev/sdc /dev/sdd /dev/sde
# or add a spare to an existing array:
sudo mdadm --manage /dev/md0 --add /dev/sde

In mdadm --detail, a spare shows as spare; after a failure it automatically transitions to spare rebuilding, then active.

Scrubbing (Data Consistency Check)

A disk can silently return wrong data (bit rot) without being marked faulty. Scrubbing reads every block and compares mirror/parity to catch mismatches before they bite during a rebuild:

echo check | sudo tee /sys/block/md0/md/sync_action   # start a scrub
cat /proc/mdstat                                       # watch progress
cat /sys/block/md0/md/mismatch_cnt                     # 0 is healthy

A non-zero mismatch_cnt is worth investigating. Most distributions ship a cron/systemd timer (mdcheck, or /etc/cron.d/mdadm) that scrubs monthly — confirm it's actually enabled in production.

Getting Notified of a Failure

RAID only helps if someone learns a disk died before the second one does. The mdadm monitor daemon watches every array and emails on a Fail/DegradedArray event:

# MAILADDR is read from /etc/mdadm/mdadm.conf
sudo mdadm --monitor --scan --daemonise
sudo mdadm --monitor --scan --test   # send a test alert to confirm delivery

On Debian/Ubuntu this runs automatically once MAILADDR [email protected] is set in mdadm.conf. Without monitoring, a degraded array can sit unnoticed for weeks — the single most common way "we had RAID" still ends in data loss.

Practical Scenario: Creating RAID 1 and Simulating a Disk Failure

Problem: a disposable test VM has two disks; the task is to build RAID 1 on them and test a disk failure.

1. Inspection (read-only):

lsblk
sudo mdadm --examine /dev/sdb /dev/sdc

Confirm both disks are empty with no old RAID metadata.

2. Create the array:

sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc
cat /proc/mdstat

Wait for the resync to finish, then confirm [UU] state.

3. Filesystem and a test file:

sudo mkfs.ext4 /dev/md0
sudo mount /dev/md0 /mnt/raid1
echo "test data" | sudo tee /mnt/raid1/test.txt

4. Simulate a disk failure:

sudo mdadm --manage /dev/md0 --fail /dev/sdb
cat /proc/mdstat
cat /mnt/raid1/test.txt

Result: /proc/mdstat shows [_U], but test.txt still reads correctly — the array keeps operating on a single disk.

5. Restore the disk:

sudo mdadm --manage /dev/md0 --remove /dev/sdb
sudo mdadm --manage /dev/md0 --add /dev/sdb
cat /proc/mdstat

Conclusion: RAID 1 keeps data continuously accessible through a disk failure, but this state needs to be detected quickly through monitoring (for example, mdadm's email notifications or mdadm --monitor) — otherwise the problem can go unnoticed until a second disk fails.

Common Mistakes

Using RAID as a Substitute for a Backup

As noted above, RAID only protects against disk failure. If a file is accidentally deleted or corrupted, it stays deleted or corrupted across every disk in the array.

Forgetting to Update mdadm.conf

If an array created manually with mdadm --create doesn't get mdadm.conf and the initramfs updated, the next reboot might not reassemble it automatically, or it might appear under a different name (/dev/md127).

Combining Disks of Different Sizes

mdadm sizes the array to match the smallest disk — extra space on larger disks goes unused. Choosing disks of matching size and speed is recommended for RAID.

Continuing to Use RAID 5 on Large, Old Disks

As disks get older and larger, the risk of a second disk failing during a rebuild increases. RAID 6 or RAID 10 are a better fit for large arrays.

Exercises

1. In a disposable VM, add three virtual disks and build a RAID 5 array on them. Compare /proc/mdstat and mdadm --detail output.
2. On both RAID 1 and RAID 5 arrays, --fail one disk and confirm data access still works.
3. Write mdadm --detail --scan output into mdadm.conf, reboot the VM, and confirm the array reassembles automatically.
4. Create an LVM PV on top of a RAID array (pvcreate /dev/md0) and add it to a VG — observe how the RAID and LVM layers work together.

Verification criterion: given a described disk failure inside a RAID array, you should be able to state the exact --fail → --remove → --add sequence needed to recover it.

Summary

Software RAID built with mdadm combines multiple disks into a logical device, providing either reliability (RAID 1/5/6/10) or speed (RAID 0) depending on the level chosen. Array state is tracked continuously through /proc/mdstat and mdadm --detail, and a failed disk is replaced in the --fail → --remove → --add order. RAID isn't a replacement for LVM but a layer added underneath it — a PV can be created on top of a RAID array, continuing with the flexible volume management covered in LVM Basics. The next step is looking at disk encryption on top of this same structure, covered in LUKS Disk Encryption.

References

• man7.org: mdadm(8)
• man7.org: mdadm.conf(5)
• man7.org: md(4) — Multiple Device driver
• kernel.org: Software RAID (Multiple Disks) support
• Red Hat: Managing RAID
• Ubuntu Server documentation: RAID