File:  [NetBSD Developer Wiki] / wikisrc / set-up_raidframe.mdwn
Revision 1.1: download - view: text, annotated - select for diffs
Fri May 13 13:25:35 2011 UTC (9 years ago) by pgoyette
Branches: MAIN
CVS tags: HEAD
Put the raidframe cookbook in the right place.

    1: [[!meta  title="Setting up raidframe(4) on NetBSD"]]
    2: 
    3: After posting lots of questions, and getting lots of expert help, I
    4: finally got my new server up and running with raid!  Since I asked
    5: so many questions, it was suggested that I recap the process so that
    6: someone else might be able to make it work without having to ask!
    7: 
    8: So ...
    9: 
   10: Here's what I did recently to get raidframe(4) up and running on my
   11: new server.
   12: 
   13: 0. This is really important!  Read the man page for raidctl(8).  It
   14:    is a long man page, but it is important to understand how things
   15:    work.  And there are some actual examples towards the end, which
   16:    will only make sense if you've read the intro material.  Don't be
   17:    intimidated by all the detail, just RTFM before you start!
   18: 
   19: 1. Use dd(1) to completely erase the drives.  This is useful to make
   20:    sure that any existing labels/wedges/etc. get removed, and also
   21:    makes it easier to initialize the raidset parity.  On Western
   22:    Digital SATA-II 500GB drive, it took less than two hours (each)
   23:    for this step.  (The operation starts off at ~110GB/sec, but will
   24:    get slower as the heads get closer to the center of the drive 
   25:    where there are fewer sectors per cylinder.)
   26: 
   27: 	dd if=/dev/zero bs=32k of=/dev/rwd2a
   28: 	dd if=/dev/zero bs=32k of=/dev/rwd3a
   29: 
   30: 2. This step might not be necessary but I wasn't taking any chances!
   31:    Also, this step is only applicable on ports which use DOS disk
   32:    layouts.  For other ports, you should be able to skip this step, 
   33:    and proceed directly to Step 3.
   34: 
   35:    Use fdisk(8) to set up DOS/Windows partition data.  Make sure you
   36:    use the "-A 2048" option for alignment.
   37: 
   38: 	fdisk -uai -A 2048 wd2a
   39: 	fdisk -uai -A 2048 wd3a
   40: 
   41:    (If you are using an older version of fdisk and get the error
   42: 
   43: 	fdisk: Bad argument to the -A flag.
   44: 
   45:    you can specify "-A 2048/2048" instead.)
   46: 
   47:    Set the partition 0 type to 169 (NetBSD), and let it occupy the
   48:    entire drive.  If you're going to boot from this raidset, you
   49:    should also make this the active partition, and update the boot
   50:    blocks when prompted.
   51: 
   52:    When finished, this is what my drive reports:
   53: 
   54: 	Partitions aligned to 16065 sector boundaries, offset 63
   55: 
   56: 	Partition table:
   57: 	0: NetBSD (sysid 169)
   58: 	    start 2048, size 976771120 (476939 MB, Cyls 0/32/33-60801/80/63),
   59: 		Active
   60: 	        PBR is not bootable: All bytes are identical (0x00)
   61: 	1: <UNUSED>
   62: 	2: <UNUSED>
   63: 	3: <UNUSED>
   64: 	Bootselector disabled.
   65: 	First active partition: 0
   66: 
   67:    (The "PBR is not bootable" line will not appear if you later use
   68:    installboot(8) - see step 12 below.)
   69: 
   70: 3. Use disklabel(8) to provide a NetBSD disklabel on each member of
   71:    the raidset.  Create an 'e' partition that describes the whole
   72:    disk - so it is essentially a copy of the 'c' partition.  Make
   73:    sure that the offset of your 'e' partition is large enough to
   74:    accomodate any architecture-specific bootstrap requirements, and
   75:    that the offset is a multiple of the hard drive's native transfer
   76:    size.
   77: 
   78:    Set the fstype on your 'e' partition to RAID (4.2BSD will also work,
   79:    but not if you want to use RAID_AUTOCONFIG).
   80: 
   81:    Here's my partition info:
   82: 
   83:    5 partitions:
   84:    #        size  offset  fstype [fsize bsize cpg/sgs]
   85:     c: 976771120    2048  unused      0     0        # (Cyl.    2*- 969020)
   86:     d: 976773168       0  unused      0     0        # (Cyl.    0 - 969020)
   87:     e: 976771120    2048    RAID                     # (Cyl.    2*- 969020)
   88: 
   89: 4: Now create a raid.conf file to describe the raidset.  Mine is a
   90:    simple RAID-1 (mirror):
   91: 
   92: 	START array
   93: 	#numrow numcol numspare
   94: 	1 2 0
   95: 
   96: 	# Identify physical disks
   97: 	START disks
   98: 	/dev/wd2e
   99: 	/dev/wd3e
  100: 
  101: 	# Layout is simple - 64 sectors per stripe
  102: 	START layout
  103: 	#Sect/StripeUnit StripeUnit/ParityUnit StripeUnit/ReconUnit RaidLevel
  104: 	128 1 1 1
  105: 
  106: 	# No spares
  107: 	#START spare
  108: 
  109: 	# Command queueing
  110: 	START queue
  111: 	fifo 100
  112: 
  113: 5. Create the raidset using raidctl(8).  Until you do this for the first
  114:    time, there are no raid "component labels" on the member drives, so you
  115:    will need to use the "-C" option;  if the raidset has been previously
  116:    created, you should use "-c" instead (this is what /etc/rc.d/raidframe
  117:    uses at system startup).
  118: 
  119: 	raidctl -C /etc/raid1.conf raid1
  120: 
  121: 6. Next, initialize the component labels with raidctl.
  122: 
  123: 	raidctl -I 201105061405 raid1
  124: 
  125:    (For serial number, I just used the current date/time; any 64-bit
  126:    integer will work.)
  127: 
  128: 7. Now use raidctl to initialize the parity of the raidset.  On my 500GB
  129:    drives, this took about 1.5 hours.
  130: 
  131: 	raidctl -i raid1
  132: 
  133:    It won't tie up your session, as the parity operation happens in the
  134:    kernel in the background.  You can monitor the progress using
  135: 
  136: 	raidctl -s raid1
  137: 
  138: 8. When parity is finished, you can treat the raidset as a normal disk.
  139:    You will need to write a NetBSD disklabel (this label is separate from
  140:    the labels on the physical drives).  Make certain that all the offsets
  141:    are multiples of the drive's native block size; otherwise you will have
  142:    unaligned transfers and performance will be very bad!  (This is also
  143:    why, in step 2 above, we use "-A 2048" for fdisk rather than letting it
  144:    default to 63!")
  145: 
  146:    Here is what my drive looks like:
  147: 
  148:    7 partitions:
  149:    #        size    offset  fstype [fsize bsize cpg/sgs]
  150:     c: 976770944         0  unused      0     0       # (Cyl.      0 - 953877*)
  151:     d: 976770944         0  unused      0     0       # (Cyl.      0 - 953877*)
  152:     e: 536870912         0  4.2BSD   2048 16384    0  # (Cyl.      0 - 524287)
  153:     f: 104857600 536870912  4.2BSD   2048 16384    0  # (Cyl. 524288 - 626687)
  154:     g: 335042432 641728512  4.2BSD   2048 16384    0  # (Cyl. 626688 - 953877*)
  155: 
  156: 9. Now, just as you would for a real disk, use newfs to initialize the
  157:    filesystems.  Then mount them and start using your new raidset!
  158: 
  159: 	newfs /dev/raid1e
  160: 	mount /dev/raid1e /mnt
  161: 
  162: 10.If your kernel includes the RAID_AUTOCONFIG option, you can use
  163:    raidctl to let your system automatically reconfigure the raidset at
  164:    every boot.  (If you don't do this, system startup will still take
  165:    care of this, as long as your configuration file is properly named
  166:    "raid[0-9].conf" or "raid[1-9][0-9].conf" and is located in /etc
  167:    directory.)
  168: 
  169: 	raidctl -A yes raid1
  170: 
  171: 11.If you're going to boot from the raidset, you probably want to have
  172:    an 'a' partition.  Here's the partition from my other, bootable,
  173:    raidset:
  174: 
  175:    6 partitions:
  176:    #        size    offset  fstype [fsize bsize cpg/sgs]
  177:     a:  41943040         0  4.2BSD   2048 16384    0  # (Cyl.      0 -  40959)
  178:     b:  62914560  41943040    swap                    # (Cyl.  40960 - 102399)
  179:     c: 488395008         0  unused      0     0       # (Cyl.      0 - 476948*)
  180:     d: 488395008         0  unused      0     0       # (Cyl.      0 - 476948*)
  181:     e: 125829120 104857600  4.2BSD   2048 16384    0  # (Cyl. 102400 - 225279)
  182:     f: 257708288 230686720  4.2BSD   2048 16384    0  # (Cyl. 225280 - 476948*)
  183: 
  184: 12.You will also need to run installboot(8) to make the raidset bootable.
  185:    Do this for each member drive:
  186: 
  187: 	mount /dev/raid0a /mnt
  188: 	installboot /dev/wd0a /usr/mdec/bootxx_ffsv1 /mnt/boot
  189: 	installboot /dev/wd1a /usr/mdec/bootxx_ffsv1 /mnt/boot
  190: 
  191: 13.Finally, you need to make sure your kernel has the RAID_AUTOCONFIG
  192:    option, and enable the raidset as the root device:
  193: 
  194: 	raidctl -A root raid0

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb