Diff for /wikisrc/guide/raidframe.mdwn between versions 1.8 and 1.13

version 1.8, 2013/03/02 13:25:13 version 1.13, 2015/06/19 19:18:31
Line 1 Line 1
   **Contents**
   
   [[!toc levels=3]]
   
 # NetBSD RAIDframe  # NetBSD RAIDframe
   
 ## RAIDframe Introduction  ## RAIDframe Introduction
Line 8  NetBSD uses the [CMU RAIDframe](http://w Line 12  NetBSD uses the [CMU RAIDframe](http://w
 its RAID subsystem. NetBSD is the primary platform for RAIDframe development.  its RAID subsystem. NetBSD is the primary platform for RAIDframe development.
 RAIDframe can also be found in older versions of FreeBSD and OpenBSD. NetBSD  RAIDframe can also be found in older versions of FreeBSD and OpenBSD. NetBSD
 also has another way of bundling disks, the  also has another way of bundling disks, the
 [ccd(4)](http://netbsd.gw.com/cgi-bin/man-cgi?ccd+4+NetBSD-5.0.1+i386) subsystem  [[!template id=man name="ccd" section="4"]] subsystem
 (see [Concatenated Disk Device](/guide/ccd)). You should possess some [basic  (see [Concatenated Disk Device](/guide/ccd)). You should possess some [basic
 knowledge](http://www.acnc.com/04_00.html) about RAID concepts and terminology  knowledge](http://www.acnc.com/04_00.html) about RAID concepts and terminology
 before continuing. You should also be at least familiar with the different  before continuing. You should also be at least familiar with the different
 levels of RAID - Adaptec provides an [excellent  levels of RAID - Adaptec provides an [excellent
 reference](http://www.adaptec.com/en-US/_common/compatibility/_education/RAID_level_compar_wp.htm),  reference](http://www.adaptec.com/en-US/_common/compatibility/_education/RAID_level_compar_wp.htm),
 and the [raid(4)](http://netbsd.gw.com/cgi-bin/man-cgi?raid+4+NetBSD-5.0.1+i386)  and the [[!template id=man name="raid" section="4"]]
 manpage contains a short overview too.  manpage contains a short overview too.
   
 ### A warning about Data Integrity, Backups, and High Availability  ### A warning about Data Integrity, Backups, and High Availability
Line 60  If you encounter problems using RAIDfram Line 64  If you encounter problems using RAIDfram
 obtaining help.  obtaining help.
   
  1. Read the RAIDframe man pages:   1. Read the RAIDframe man pages:
     [raid(4)](http://netbsd.gw.com/cgi-bin/man-cgi?raid+4+NetBSD-5.0.1+i386) and      [[!template id=man name="raid" section="4"]] and
     [raidctl(8)](http://netbsd.gw.com/cgi-bin/man-cgi?raidctl+8+NetBSD-5.0.1+i386)      [[!template id=man name="raidctl" section="8"]]
     thoroughly.      thoroughly.
   
  2. Search the mailing list archives. Unfortunately, there is no NetBSD list   2. Search the mailing list archives. Unfortunately, there is no NetBSD list
Line 82  obtaining help. Line 86  obtaining help.
  4. If your problem persists: Post to the mailing list most appropriate   4. If your problem persists: Post to the mailing list most appropriate
     (judgment call). Collect as much verbosely detailed information as possible      (judgment call). Collect as much verbosely detailed information as possible
     before posting: Include your      before posting: Include your
     [dmesg(8)](http://netbsd.gw.com/cgi-bin/man-cgi?dmesg+8+NetBSD-5.0.1+i386)      [[!template id=man name="dmesg" section="8"]]
     output from `/var/run/dmesg.boot`, your kernel      output from `/var/run/dmesg.boot`, your kernel
     [config(5)](http://netbsd.gw.com/cgi-bin/man-cgi?config+5+NetBSD-5.0.1+i386) ,      [[!template id=man name="config" section="5"]] ,
     your `/etc/raid[0-9].conf`, any relevant errors on `/dev/console`,      your `/etc/raid[0-9].conf`, any relevant errors on `/dev/console`,
     `/var/log/messages`, or to `stdout/stderr` of      `/var/log/messages`, or to `stdout/stderr` of
     [raidctl(8)](http://netbsd.gw.com/cgi-bin/man-cgi?raidctl+8+NetBSD-5.0.1+i386).      [[!template id=man name="raidctl" section="8"]].
     The output of **raidctl -s** (if available) will be useful as well. Also      The output of **raidctl -s** (if available) will be useful as well. Also
     include details on the troubleshooting steps you've taken thus far, exactly      include details on the troubleshooting steps you've taken thus far, exactly
     when the problem started, and any notes on recent changes that may have      when the problem started, and any notes on recent changes that may have
Line 109  options: Line 113  options:
   
 The RAID support must be detected by the NetBSD kernel, which can be checked by  The RAID support must be detected by the NetBSD kernel, which can be checked by
 looking at the output of the  looking at the output of the
 [dmesg(8)](http://netbsd.gw.com/cgi-bin/man-cgi?dmesg+8+NetBSD-5.0.1+i386)  [[!template id=man name="dmesg" section="8"]]
 command.  command.
   
     # dmesg|grep -i raid      # dmesg|grep -i raid
Line 132  redundant power, this will improve drive Line 136  redundant power, this will improve drive
 redundant power, the write cache could endanger the integrity of RAID data in  redundant power, the write cache could endanger the integrity of RAID data in
 the event of a power loss.  the event of a power loss.
   
 The [dkctl(8)](http://netbsd.gw.com/cgi-bin/man-cgi?dkctl+8+NetBSD-5.0.1+i386)  The [[!template id=man name="dkctl" section="8"]]
 utility to can be used for this on all kinds of disks that support the operation  utility to can be used for this on all kinds of disks that support the operation
 (SCSI, EIDE, SATA, ...):  (SCSI, EIDE, SATA, ...):
   
Line 162  the RAID and: Line 166  the RAID and:
    be able to quickly reconfigure the system to boot from the remaining     be able to quickly reconfigure the system to boot from the remaining
    component (platform dependent).     component (platform dependent).
   
 ![RAID-1 Disk Logical Layout](/guide/images/raidframe_raidl1-diskdia.png)  ![RAID-1 Disk Logical Layout](/guide/images/raidframe_raidl1-diskdia.png)  
   
 **RAID-1 Disk Logical Layout**  **RAID-1 Disk Logical Layout**
   
 Because RAID-1 provides both redundancy and performance improvements, its most  Because RAID-1 provides both redundancy and performance improvements, its most
Line 191  has become the de facto standard for set Line 194  has become the de facto standard for set
  1. Install a stock NetBSD onto Disk0 of your system.   1. Install a stock NetBSD onto Disk0 of your system.
   
   
     ![Perform generic install onto Disk0/wd0](/guide/images/raidframe_r1r-pp1.png)      ![Perform generic install onto Disk0/wd0](/guide/images/raidframe_r1r-pp1.png)  
   
     **Perform generic install onto Disk0/wd0**      **Perform generic install onto Disk0/wd0**
   
  2. Use the installed system on Disk0/wd0 to setup a RAID Set composed of   2. Use the installed system on Disk0/wd0 to setup a RAID Set composed of
     Disk1/wd1 only.      Disk1/wd1 only.
   
     ![Setup RAID Set](/guide/images/raidframe_r1r-pp2.png)      ![Setup RAID Set](/guide/images/raidframe_r1r-pp2.png)  
   
     **Setup RAID Set**      **Setup RAID Set**
   
  3. Reboot the system off the Disk1/wd1 with the newly created RAID volume.   3. Reboot the system off the Disk1/wd1 with the newly created RAID volume.
   
   
     ![Reboot using Disk1/wd1 of RAID](/guide/images/raidframe_r1r-pp3.png)      ![Reboot using Disk1/wd1 of RAID](/guide/images/raidframe_r1r-pp3.png)  
   
     **Reboot using Disk1/wd1 of RAID**      **Reboot using Disk1/wd1 of RAID**
   
   
  4. Add/re-sync Disk0/wd0 back into the RAID set.   4. Add/re-sync Disk0/wd0 back into the RAID set.
   
     ![Mirror Disk1/wd1 back to Disk0/wd0](/guide/images/raidframe_r1r-pp4.png)      ![Mirror Disk1/wd1 back to Disk0/wd0](/guide/images/raidframe_r1r-pp4.png)  
   
     **Mirror Disk1/wd1 back to Disk0/wd0**      **Mirror Disk1/wd1 back to Disk0/wd0**
   
 ### Hardware Review  ### Hardware Review
Line 286  the entire disk for NetBSD`, answer `yes Line 285  the entire disk for NetBSD`, answer `yes
  * [NetBSD/sparc64 Install](http://ftp.NetBSD.org/pub/NetBSD/NetBSD-5.0.2/sparc64/INSTALL.html)   * [NetBSD/sparc64 Install](http://ftp.NetBSD.org/pub/NetBSD/NetBSD-5.0.2/sparc64/INSTALL.html)
   
 Once the installation is complete, you should examine the  Once the installation is complete, you should examine the
 [disklabel(8)](http://netbsd.gw.com/cgi-bin/man-cgi?disklabel+8+NetBSD-5.0.1+i386)  [[!template id=man name="disklabel" section="8"]]
 and [fdisk(8)](http://netbsd.gw.com/cgi-bin/man-cgi?fdisk+8+NetBSD-5.0.1+i386) /  and [[!template id=man name="fdisk" section="8"]] /
 [sunlabel(8)](http://netbsd.gw.com/cgi-bin/man-cgi?sunlabel+8+NetBSD-5.0.1+i386)  [[!template id=man name="sunlabel" section="8"]]
 outputs on the system:  outputs on the system:
   
     # df      # df
Line 365  On Sparc64 the command and output differ Line 364  On Sparc64 the command and output differ
 Once you have a stock install of NetBSD on Disk0/wd0, you are ready to begin.  Once you have a stock install of NetBSD on Disk0/wd0, you are ready to begin.
 Disk1/wd1 will be visible and unused by the system. To setup Disk1/wd1, you will  Disk1/wd1 will be visible and unused by the system. To setup Disk1/wd1, you will
 use  use
 [disklabel(8)](http://netbsd.gw.com/cgi-bin/man-cgi?disklabel+8+NetBSD-5.0.1+i386)  [[!template id=man name="disklabel" section="8"]]
 to allocate the entire second disk to the RAID-1 set.  to allocate the entire second disk to the RAID-1 set.
   
 *Tip*: The best way to ensure that Disk1/wd1 is completely empty is to 'zero'  *Tip*:
 out the first few sectors of the disk with  > The best way to ensure that Disk1/wd1 is completely empty is to 'zero'
 [dd(1)](http://netbsd.gw.com/cgi-bin/man-cgi?dd+1+NetBSD-5.0.1+i386) . This will  > out the first few sectors of the disk with
 erase the MBR (i386) or Sun disk label (sparc64), as well as the NetBSD disk  > [[!template id=man name="dd" section="1"]] . This will
 label. If you make a mistake at any point during the RAID setup process, you can  > erase the MBR (i386) or Sun disk label (sparc64), as well as the NetBSD disk
 always refer to this process to restore the disk to an empty state.  > label. If you make a mistake at any point during the RAID setup process, you can
   > always refer to this process to restore the disk to an empty state.
 *Note*: On sparc64, use `/dev/rwd1c` instead of `/dev/rwd1d`!  > 
   > *Note*: On sparc64, use `/dev/rwd1c` instead of `/dev/rwd1d`!
     # dd if=/dev/zero of=/dev/rwd1d bs=8k count=1  > 
     1+0 records in  >     # dd if=/dev/zero of=/dev/rwd1d bs=8k count=1
     1+0 records out  >     1+0 records in
     8192 bytes transferred in 0.003 secs (2730666 bytes/sec)  >     1+0 records out
   >     8192 bytes transferred in 0.003 secs (2730666 bytes/sec)
 Once this is complete, on i386, verify that both the MBR and NetBSD disk labels  > 
 are gone. On sparc64, verify that the Sun Disk label is gone as well.  > Once this is complete, on i386, verify that both the MBR and NetBSD disk labels
   > are gone. On sparc64, verify that the Sun Disk label is gone as well.
 On i386:  > 
   > On i386:
     # fdisk /dev/rwd1d  > 
       >     # fdisk /dev/rwd1d
     fdisk: primary partition table invalid, no magic in sector 0  >     
     Disk: /dev/rwd1d  >     fdisk: primary partition table invalid, no magic in sector 0
     NetBSD disklabel disk geometry:  >     Disk: /dev/rwd1d
     cylinders: 19386, heads: 16, sectors/track: 63 (1008 sectors/cylinder)  >     NetBSD disklabel disk geometry:
     total sectors: 19541088  >     cylinders: 19386, heads: 16, sectors/track: 63 (1008 sectors/cylinder)
       >     total sectors: 19541088
     BIOS disk geometry:  >     
     cylinders: 1023, heads: 255, sectors/track: 63 (16065 sectors/cylinder)  >     BIOS disk geometry:
     total sectors: 19541088  >     cylinders: 1023, heads: 255, sectors/track: 63 (16065 sectors/cylinder)
       >     total sectors: 19541088
     Partition table:  >     
     0: <UNUSED>  >     Partition table:
     1: <UNUSED>  >     0: <UNUSED>
     2: <UNUSED>  >     1: <UNUSED>
     3: <UNUSED>  >     2: <UNUSED>
     Bootselector disabled.  >     3: <UNUSED>
       >     Bootselector disabled.
     # disklabel -r wd1  >     
       >     # disklabel -r wd1
     [...snip...]  >     
     16 partitions:  >     [...snip...]
     #        size    offset     fstype [fsize bsize cpg/sgs]  >     16 partitions:
      c:  19541025        63     unused      0     0        # (Cyl.      0* - 19385)  >     #        size    offset     fstype [fsize bsize cpg/sgs]
      d:  19541088         0     unused      0     0        # (Cyl.      0 - 19385)  >      c:  19541025        63     unused      0     0        # (Cyl.      0* - 19385)
   >      d:  19541088         0     unused      0     0        # (Cyl.      0 - 19385)
 On sparc64:  > 
   > On sparc64:
     # sunlabel /dev/rwd1c  > 
       >     # sunlabel /dev/rwd1c
     sunlabel: bogus label on `/dev/wd1c' (bad magic number)  >     
       >     sunlabel: bogus label on `/dev/wd1c' (bad magic number)
     # disklabel -r wd1  >     
       >     # disklabel -r wd1
     [...snip...]  >     
     3 partitions:  >     [...snip...]
     #        size    offset     fstype [fsize bsize cpg/sgs]  >     3 partitions:
      c:  19541088         0     unused      0     0        # (Cyl.      0 -  19385)  >     #        size    offset     fstype [fsize bsize cpg/sgs]
     disklabel: boot block size 0  >      c:  19541088         0     unused      0     0        # (Cyl.      0 -  19385)
     disklabel: super block size 0  >     disklabel: boot block size 0
   >     disklabel: super block size 0
   
 Now that you are certain the second disk is empty, on i386 you must establish  Now that you are certain the second disk is empty, on i386 you must establish
 the MBR on the second disk using the values obtained from Disk0/wd0 above. We  the MBR on the second disk using the values obtained from Disk0/wd0 above. We
 must remember to mark the NetBSD partition active or the system will not boot.  must remember to mark the NetBSD partition active or the system will not boot.
 You must also create a NetBSD disklabel on Disk1/wd1 that will enable a RAID  You must also create a NetBSD disklabel on Disk1/wd1 that will enable a RAID
 volume to exist upon it. On sparc64, you will need to simply  volume to exist upon it. On sparc64, you will need to simply
 [disklabel(8)](http://netbsd.gw.com/cgi-bin/man-cgi?disklabel+8+NetBSD-5.0.1+i386)  [[!template id=man name="disklabel" section="8"]]
 the second disk which will write the proper Sun Disk Label.  the second disk which will write the proper Sun Disk Label.
   
 *Tip*:  *Tip*:
 [disklabel(8)](http://netbsd.gw.com/cgi-bin/man-cgi?disklabel+8+NetBSD-5.0.1+i386)  [[!template id=man name="disklabel" section="8"]]
 will use your shell' s environment variable `$EDITOR` variable to edit the  will use your shell' s environment variable `$EDITOR` variable to edit the
 disklabel. The default is  disklabel. The default is
 [vi(1)](http://netbsd.gw.com/cgi-bin/man-cgi?vi+1+NetBSD-5.0.1+i386)  [[!template id=man name="vi" section="1"]]
   
 On i386:  On i386:
   
Line 523  On sparc64: Line 523  On sparc64:
     a: start cyl =      0, size = 19541088 (19386/0/0 - 9541.55Mb)      a: start cyl =      0, size = 19541088 (19386/0/0 - 9541.55Mb)
     c: start cyl =      0, size = 19541088 (19386/0/0 - 9541.55Mb)      c: start cyl =      0, size = 19541088 (19386/0/0 - 9541.55Mb)
   
 *Note*: On i386, the `c:` and `d:` slices are reserved. `c:` represents the  *Note*: On i386, the `c:` and `d:` slices are reserved. `c:` represents the 
 NetBSD portion of the disk. `d:` represents the entire disk. Because we want to  NetBSD portion of the disk. `d:` represents the entire disk. Because we want to 
 allocate the entire NetBSD MBR partition to RAID, and because `a:` resides  allocate the entire NetBSD MBR partition to RAID, and because `a:` resides 
 within the bounds of `c:`, the `a:` and `c:` slices have same size and offset  within the bounds of `c:`, the `a:` and `c:` slices have same size and offset 
 values and sizes. The offset must start at a track boundary (an increment of  values. The offset must start at a track boundary (an increment of sectors 
 sectors matching the sectors/track value in the disk label). On sparc64 however,  matching the sectors/track value in the disk label). On sparc64 however, `c:` 
 `c:` represents the entire NetBSD partition in the Sun disk label and `d:` is  represents the entire NetBSD partition in the Sun disk label and `d:` is not 
 not reserved. Also note that sparc64's `c:` and `a:` require no offset from the  reserved. Also note that sparc64's `c:` and `a:` require no offset from the 
 beginning of the disk, however if they should need to be, the offset must start  beginning of the disk, however if they should need to be, the offset must start 
 at a cylinder boundary (an increment of sectors matching the sectors/cylinder  at a cylinder boundary (an increment of sectors matching the sectors/cylinder 
 value).  value).
   
 ### Initializing the RAID Device  ### Initializing the RAID Device
Line 705  Next, format the newly created `/` parti Line 705  Next, format the newly created `/` parti
   
 The new RAID filesystems are now ready for use. We mount them under `/mnt` and  The new RAID filesystems are now ready for use. We mount them under `/mnt` and
 copy all files from the old system. This can be done using  copy all files from the old system. This can be done using
 [dump(8)](http://netbsd.gw.com/cgi-bin/man-cgi?dump+8+NetBSD-5.0.1+i386) or  [[!template id=man name="dump" section="8"]] or
 [pax(1)](http://netbsd.gw.com/cgi-bin/man-cgi?pax+1+NetBSD-5.0.1+i386).  [[!template id=man name="pax" section="1"]].
   
     # mount /dev/raid0a /mnt      # mount /dev/raid0a /mnt
     # df -h /mnt      # df -h /mnt
Line 719  The NetBSD install now exists on the RAI Line 719  The NetBSD install now exists on the RAI
 mount-points in the new copy of `/etc/fstab` or the system will not come up  mount-points in the new copy of `/etc/fstab` or the system will not come up
 correctly. Replace instances of `wd0` with `raid0`.  correctly. Replace instances of `wd0` with `raid0`.
   
       # mv /mnt/etc/fstab /mnt/etc/fstab.old
       # sed 's/wd0/raid0/g' /mnt/etc/fstab.old > /mnt/etc/fstab
   
 The swap should be unconfigured upon shutdown to avoid parity errors on the RAID  The swap should be unconfigured upon shutdown to avoid parity errors on the RAID
 device. This can be done with a simple, one-line setting in `/etc/rc.conf`.  device. This can be done with a simple, one-line setting in `/etc/rc.conf`.
   
Line 737  from which disk the system is booting. Line 740  from which disk the system is booting.
   
 *Caution*: Although it may seem logical to install the 1st stage boot block into  *Caution*: Although it may seem logical to install the 1st stage boot block into
 `/dev/rwd1{c,d}` (which is historically correct with NetBSD 1.6.x  `/dev/rwd1{c,d}` (which is historically correct with NetBSD 1.6.x
 [installboot(8)](http://netbsd.gw.com/cgi-bin/man-cgi?installboot+8+NetBSD-5.0.1+i386)  [[!template id=man name="installboot" section="8"]]
 , this is no longer the case. If you make this mistake, the boot sector will  , this is no longer the case. If you make this mistake, the boot sector will
 become irrecoverably damaged and you will need to start the process over again.  become irrecoverably damaged and you will need to start the process over again.
   
Line 777  rebooted. After the reboot everything is Line 780  rebooted. After the reboot everything is
     # shutdown -r now      # shutdown -r now
   
 *Warning*: Always use  *Warning*: Always use
 [shutdown(8)](http://netbsd.gw.com/cgi-bin/man-cgi?shutdown+8+NetBSD-5.0.1+i386)  [[!template id=man name="shutdown" section="8"]]
 when shutting down. Never simply use  when shutting down. Never simply use
 [reboot(8)](http://netbsd.gw.com/cgi-bin/man-cgi?reboot+8+NetBSD-5.0.1+i386).  [[!template id=man name="reboot" section="8"]].
 [reboot(8)](http://netbsd.gw.com/cgi-bin/man-cgi?reboot+8+NetBSD-5.0.1+i386)  [[!template id=man name="reboot" section="8"]]
 will not properly run shutdown RC scripts and will not safely disable swap. This  will not properly run shutdown RC scripts and will not safely disable swap. This
 will cause dirty parity at every reboot.  will cause dirty parity at every reboot.
   
Line 843  effectively bringing Disk0/wd0 into the  Line 846  effectively bringing Disk0/wd0 into the 
     # disklabel -R -r wd0 /tmp/disklabel.wd1      # disklabel -R -r wd0 /tmp/disklabel.wd1
   
 As a last-minute sanity check, you might want to use  As a last-minute sanity check, you might want to use
 [diff(1)](http://netbsd.gw.com/cgi-bin/man-cgi?diff+1+NetBSD-5.0.1+i386) to  [[!template id=man name="diff" section="1"]] to
 ensure that the disklabels of Disk0/wd0 match Disk1/wd1. You should also backup  ensure that the disklabels of Disk0/wd0 match Disk1/wd1. You should also backup
 these files for reference in the event of an emergency.  these files for reference in the event of an emergency.
   
Line 936  priority to Disk1/wd1 before Disk0/wd0.  Line 939  priority to Disk1/wd1 before Disk0/wd0. 
   
 BIOS.  BIOS.
   
 ![Award BIOS i386 Boot Disk1/wd1](/guide/images/raidframe_awardbios2.png)  ![Award BIOS i386 Boot Disk1/wd1](/guide/images/raidframe_awardbios2.png)  
   
 **Award BIOS i386 Boot Disk1/wd1**  **Award BIOS i386 Boot Disk1/wd1**
   
 Save changes and exit:  Save changes and exit:
Line 953  You can determine that the BIOS is readi Line 955  You can determine that the BIOS is readi
 boot loader is 30 seconds instead of 15. After the reboot, re-enter the BIOS an  boot loader is 30 seconds instead of 15. After the reboot, re-enter the BIOS an
 configure the drive boot order back to the default:  configure the drive boot order back to the default:
   
 ![Award BIOS i386 Boot Disk0/wd0](/guide/images/raidframe_awardbios1.png)  ![Award BIOS i386 Boot Disk0/wd0](/guide/images/raidframe_awardbios1.png)  
   
 **Award BIOS i386 Boot Disk0/wd0**  **Award BIOS i386 Boot Disk0/wd0**
   
 Save changes and exit:  Save changes and exit:
Line 1019  And the second disk: Line 1020  And the second disk:
     [...snip...]      [...snip...]
   
 At each boot, the following should appear in the NetBSD kernel  At each boot, the following should appear in the NetBSD kernel
 [dmesg(8)](http://netbsd.gw.com/cgi-bin/man-cgi?dmesg+8+NetBSD-5.0.1+i386) :  [[!template id=man name="dmesg" section="8"]] :
   
     Kernelized RAIDframe activated      Kernelized RAIDframe activated
     raid0: RAID Level 1      raid0: RAID Level 1

Removed from v.1.8  
changed lines
  Added in v.1.13


CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb