Annotation of wikisrc/set-up_raidframe.mdwn, revision 1.1

1.1     ! pgoyette    1: [[!meta  title="Setting up raidframe(4) on NetBSD"]]
        !             2: 
        !             3: After posting lots of questions, and getting lots of expert help, I
        !             4: finally got my new server up and running with raid!  Since I asked
        !             5: so many questions, it was suggested that I recap the process so that
        !             6: someone else might be able to make it work without having to ask!
        !             7: 
        !             8: So ...
        !             9: 
        !            10: Here's what I did recently to get raidframe(4) up and running on my
        !            11: new server.
        !            12: 
        !            13: 0. This is really important!  Read the man page for raidctl(8).  It
        !            14:    is a long man page, but it is important to understand how things
        !            15:    work.  And there are some actual examples towards the end, which
        !            16:    will only make sense if you've read the intro material.  Don't be
        !            17:    intimidated by all the detail, just RTFM before you start!
        !            18: 
        !            19: 1. Use dd(1) to completely erase the drives.  This is useful to make
        !            20:    sure that any existing labels/wedges/etc. get removed, and also
        !            21:    makes it easier to initialize the raidset parity.  On Western
        !            22:    Digital SATA-II 500GB drive, it took less than two hours (each)
        !            23:    for this step.  (The operation starts off at ~110GB/sec, but will
        !            24:    get slower as the heads get closer to the center of the drive 
        !            25:    where there are fewer sectors per cylinder.)
        !            26: 
        !            27:        dd if=/dev/zero bs=32k of=/dev/rwd2a
        !            28:        dd if=/dev/zero bs=32k of=/dev/rwd3a
        !            29: 
        !            30: 2. This step might not be necessary but I wasn't taking any chances!
        !            31:    Also, this step is only applicable on ports which use DOS disk
        !            32:    layouts.  For other ports, you should be able to skip this step, 
        !            33:    and proceed directly to Step 3.
        !            34: 
        !            35:    Use fdisk(8) to set up DOS/Windows partition data.  Make sure you
        !            36:    use the "-A 2048" option for alignment.
        !            37: 
        !            38:        fdisk -uai -A 2048 wd2a
        !            39:        fdisk -uai -A 2048 wd3a
        !            40: 
        !            41:    (If you are using an older version of fdisk and get the error
        !            42: 
        !            43:        fdisk: Bad argument to the -A flag.
        !            44: 
        !            45:    you can specify "-A 2048/2048" instead.)
        !            46: 
        !            47:    Set the partition 0 type to 169 (NetBSD), and let it occupy the
        !            48:    entire drive.  If you're going to boot from this raidset, you
        !            49:    should also make this the active partition, and update the boot
        !            50:    blocks when prompted.
        !            51: 
        !            52:    When finished, this is what my drive reports:
        !            53: 
        !            54:        Partitions aligned to 16065 sector boundaries, offset 63
        !            55: 
        !            56:        Partition table:
        !            57:        0: NetBSD (sysid 169)
        !            58:            start 2048, size 976771120 (476939 MB, Cyls 0/32/33-60801/80/63),
        !            59:                Active
        !            60:                PBR is not bootable: All bytes are identical (0x00)
        !            61:        1: <UNUSED>
        !            62:        2: <UNUSED>
        !            63:        3: <UNUSED>
        !            64:        Bootselector disabled.
        !            65:        First active partition: 0
        !            66: 
        !            67:    (The "PBR is not bootable" line will not appear if you later use
        !            68:    installboot(8) - see step 12 below.)
        !            69: 
        !            70: 3. Use disklabel(8) to provide a NetBSD disklabel on each member of
        !            71:    the raidset.  Create an 'e' partition that describes the whole
        !            72:    disk - so it is essentially a copy of the 'c' partition.  Make
        !            73:    sure that the offset of your 'e' partition is large enough to
        !            74:    accomodate any architecture-specific bootstrap requirements, and
        !            75:    that the offset is a multiple of the hard drive's native transfer
        !            76:    size.
        !            77: 
        !            78:    Set the fstype on your 'e' partition to RAID (4.2BSD will also work,
        !            79:    but not if you want to use RAID_AUTOCONFIG).
        !            80: 
        !            81:    Here's my partition info:
        !            82: 
        !            83:    5 partitions:
        !            84:    #        size  offset  fstype [fsize bsize cpg/sgs]
        !            85:     c: 976771120    2048  unused      0     0        # (Cyl.    2*- 969020)
        !            86:     d: 976773168       0  unused      0     0        # (Cyl.    0 - 969020)
        !            87:     e: 976771120    2048    RAID                     # (Cyl.    2*- 969020)
        !            88: 
        !            89: 4: Now create a raid.conf file to describe the raidset.  Mine is a
        !            90:    simple RAID-1 (mirror):
        !            91: 
        !            92:        START array
        !            93:        #numrow numcol numspare
        !            94:        1 2 0
        !            95: 
        !            96:        # Identify physical disks
        !            97:        START disks
        !            98:        /dev/wd2e
        !            99:        /dev/wd3e
        !           100: 
        !           101:        # Layout is simple - 64 sectors per stripe
        !           102:        START layout
        !           103:        #Sect/StripeUnit StripeUnit/ParityUnit StripeUnit/ReconUnit RaidLevel
        !           104:        128 1 1 1
        !           105: 
        !           106:        # No spares
        !           107:        #START spare
        !           108: 
        !           109:        # Command queueing
        !           110:        START queue
        !           111:        fifo 100
        !           112: 
        !           113: 5. Create the raidset using raidctl(8).  Until you do this for the first
        !           114:    time, there are no raid "component labels" on the member drives, so you
        !           115:    will need to use the "-C" option;  if the raidset has been previously
        !           116:    created, you should use "-c" instead (this is what /etc/rc.d/raidframe
        !           117:    uses at system startup).
        !           118: 
        !           119:        raidctl -C /etc/raid1.conf raid1
        !           120: 
        !           121: 6. Next, initialize the component labels with raidctl.
        !           122: 
        !           123:        raidctl -I 201105061405 raid1
        !           124: 
        !           125:    (For serial number, I just used the current date/time; any 64-bit
        !           126:    integer will work.)
        !           127: 
        !           128: 7. Now use raidctl to initialize the parity of the raidset.  On my 500GB
        !           129:    drives, this took about 1.5 hours.
        !           130: 
        !           131:        raidctl -i raid1
        !           132: 
        !           133:    It won't tie up your session, as the parity operation happens in the
        !           134:    kernel in the background.  You can monitor the progress using
        !           135: 
        !           136:        raidctl -s raid1
        !           137: 
        !           138: 8. When parity is finished, you can treat the raidset as a normal disk.
        !           139:    You will need to write a NetBSD disklabel (this label is separate from
        !           140:    the labels on the physical drives).  Make certain that all the offsets
        !           141:    are multiples of the drive's native block size; otherwise you will have
        !           142:    unaligned transfers and performance will be very bad!  (This is also
        !           143:    why, in step 2 above, we use "-A 2048" for fdisk rather than letting it
        !           144:    default to 63!")
        !           145: 
        !           146:    Here is what my drive looks like:
        !           147: 
        !           148:    7 partitions:
        !           149:    #        size    offset  fstype [fsize bsize cpg/sgs]
        !           150:     c: 976770944         0  unused      0     0       # (Cyl.      0 - 953877*)
        !           151:     d: 976770944         0  unused      0     0       # (Cyl.      0 - 953877*)
        !           152:     e: 536870912         0  4.2BSD   2048 16384    0  # (Cyl.      0 - 524287)
        !           153:     f: 104857600 536870912  4.2BSD   2048 16384    0  # (Cyl. 524288 - 626687)
        !           154:     g: 335042432 641728512  4.2BSD   2048 16384    0  # (Cyl. 626688 - 953877*)
        !           155: 
        !           156: 9. Now, just as you would for a real disk, use newfs to initialize the
        !           157:    filesystems.  Then mount them and start using your new raidset!
        !           158: 
        !           159:        newfs /dev/raid1e
        !           160:        mount /dev/raid1e /mnt
        !           161: 
        !           162: 10.If your kernel includes the RAID_AUTOCONFIG option, you can use
        !           163:    raidctl to let your system automatically reconfigure the raidset at
        !           164:    every boot.  (If you don't do this, system startup will still take
        !           165:    care of this, as long as your configuration file is properly named
        !           166:    "raid[0-9].conf" or "raid[1-9][0-9].conf" and is located in /etc
        !           167:    directory.)
        !           168: 
        !           169:        raidctl -A yes raid1
        !           170: 
        !           171: 11.If you're going to boot from the raidset, you probably want to have
        !           172:    an 'a' partition.  Here's the partition from my other, bootable,
        !           173:    raidset:
        !           174: 
        !           175:    6 partitions:
        !           176:    #        size    offset  fstype [fsize bsize cpg/sgs]
        !           177:     a:  41943040         0  4.2BSD   2048 16384    0  # (Cyl.      0 -  40959)
        !           178:     b:  62914560  41943040    swap                    # (Cyl.  40960 - 102399)
        !           179:     c: 488395008         0  unused      0     0       # (Cyl.      0 - 476948*)
        !           180:     d: 488395008         0  unused      0     0       # (Cyl.      0 - 476948*)
        !           181:     e: 125829120 104857600  4.2BSD   2048 16384    0  # (Cyl. 102400 - 225279)
        !           182:     f: 257708288 230686720  4.2BSD   2048 16384    0  # (Cyl. 225280 - 476948*)
        !           183: 
        !           184: 12.You will also need to run installboot(8) to make the raidset bootable.
        !           185:    Do this for each member drive:
        !           186: 
        !           187:        mount /dev/raid0a /mnt
        !           188:        installboot /dev/wd0a /usr/mdec/bootxx_ffsv1 /mnt/boot
        !           189:        installboot /dev/wd1a /usr/mdec/bootxx_ffsv1 /mnt/boot
        !           190: 
        !           191: 13.Finally, you need to make sure your kernel has the RAID_AUTOCONFIG
        !           192:    option, and enable the raidset as the root device:
        !           193: 
        !           194:        raidctl -A root raid0

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb