Saturday, March 20, 2010

configuring RAID 1 array and replacing failed device

This post will explain how to configure raid1 on two identical HDD using debian. and in case of a HDD failure how to add a new HDD.

First we assume you have booted form /dev/sad HDD and sda have sda1,sda2,sda3 as ext3 partitions and sda4 is swap.

Configuring RAID 1 array

Raid array will be manage through mdadm package. First need to install mdadn.
* apt-get install mdadm

make sure kernel modules will be loaded in the boot time
* add "md" and "raid1" modules to the /etc/modules file

load the modules.
* modprobe raid1
* modprobe md

Using lsmod command you can see the loaded modules

now your configuration install in the /dev/sda. You can copy the partition table to the second drive(/dev/sdb)
* sfdisk -d /dev/sda | sfdisk /dev/sdb

First start configuring raid in the second drive. Because once you format the raid array the data will be lost and in case of a reboot you will be facing problem on mounting devises. There fore first Change the /dev/sdb partition types in to fd which is Linux raid autodetect type.
* sfdisk --change-id /dev/sdb 1 fd
* sfdisk --change-id /dev/sdb 2 fd
* sfdisk --change-id /dev/sdb 3 fd

Create the raid1 array and add the sdb devise. level=1 for raid1
* mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb1
* mdadm --create /dev/md2 --level=1 --raid-disks=2 missing /dev/sdb2
* mdadm --create /dev/md3 --level=1 --raid-disks=2 missing /dev/sdb3

now format the array
* mkfs.ext3 /dev/md1
* mkfs.ext3 /dev/md2
* mkfs.ext3 /dev/md3

you need to mount the array device every time boot the edit the /etc/fstab file
* proc /proc proc defaults 0 0
/dev/md1 / ext3 defaults 0 1
/dev/md2 /home ext3 defaults 0 2
/dev/md3 /mnt ext3 defaults 0 2

note above devices are /dev/md1, /dev/md2, /dev/md3 instead of physical drives

configure the grub

* sed 's/sda1/md1/' < /boot/grub/menu.lst_orig > /boot/grub/menu.lst
* update-grub

reboot from single mode
* init 1

mount the array and sync all the data on /dev/sda
* mount /dev/md1 /media
* rsync -aqxP / /media
* umount /media

* mount /dev/md2 /media
* rsync -aqxP /home /media
* umount /media

* mount /dev/md3 /media
* rsync -aqxP /mnt /media
* umount /media

then boot from the /dev/sdb, setup the boot manager
* grub
* device (hd0) /dev/sdb
* root (hd0,0)
* setup (hd0)
* quit

reboot the machine add the sda.
* sfdisk --change-id /dev/sda 1 fd
* sfdisk --change-id /dev/sda 2 fd
* sfdisk --change-id /dev/sda 3 fd

* mdadm --add /dev/md1 /dev/sda1
* mdadm --add /dev/md2 /dev/sda2
* mdadm --add /dev/md3 /dev/sda3

check the status
* watch cat /proc/mdstat

comment the last three lines of /etc/mdadm/mdadm.conf file and add the following again
* mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Replace failed HDD
identify the failed drive
* cat /proc/mdstat

first mark the device and then remove from the array
* mdadm --manage /dev/md1 --fail /dev/sdb1
* mdadm --manage /dev/md1 --remove /dev/sdb1

* mdadm --manage /dev/md2 --fail /dev/sdb1
* mdadm --manage /dev/md2 --remove /dev/sdb1

* mdadm --manage /dev/md3 --fail /dev/sdb1
* mdadm --manage /dev/md3 --remove /dev/sdb1

shut down the machine and start after adding the new drive.Then cop the partition table.
* sfdisk -d /dev/sda | sfdisk /dev/sdb

again add the new device to the array
* mdadm --manage /dev/md1 --add /dev/sdb1
* mdadm --manage /dev/md2 --add /dev/sdb2
* mdadm --manage /dev/md3 --add /dev/sdb3