File:  [NetBSD Developer Wiki] / wikisrc / guide / ccd.mdwn
Revision 1.5: download - view: text, annotated - select for diffs
Fri Jun 19 19:18:31 2015 UTC (5 years ago) by plunky
Branches: MAIN
CVS tags: HEAD
replace direct links to manpages on netbsd.gw.com with templates

    1: **Contents**
    2: 
    3: [[!toc levels=3]]
    4: 
    5: # Concatenated Disk Device (CCD) configuration
    6: 
    7: The CCD driver allows the user to *concatenate* several physical disks into one
    8: pseudo volume. While RAIDframe (see [[NetBSD RAIDframe|guide/rf]]) also allows
    9: doing this to create RAID level 0 sets, it does not allow you to do striping
   10: across disks of different geometry, which is where CCD comes in handy. CCD also
   11: allows for an *interleave* to improve disk performance with a gained space loss.
   12: This example will not cover that feature.
   13: 
   14: The steps required to setup a CCD are as follows:
   15: 
   16:  1. Install physical media
   17:  2. Configure kernel support
   18:  3. Disklabel each volume member of the CCD
   19:  4. Configure the CCD conf file
   20:  5. Initialize the CCD device
   21:  6. Create a filesystem on the new CCD device
   22:  7. Mount the CCD filesystem
   23: 
   24: This example features a CCD setup on NetBSD/sparc 1.5. The CCD will reside on 4
   25: SCSI disks in a generic external Sun disk pack chassis connected to the external
   26: 50 pin SCSI port.
   27: 
   28: ## Install physical media
   29: 
   30: This step is at your own discretion, depending on your platform and the hardware
   31: at your disposal.
   32: 
   33: From my DMESG:
   34: 
   35:     Disk #1:
   36:       probe(esp0:0:0): max sync rate 10.00MB/s
   37:       sd0 at scsibus0 target 0 lun 0: <SEAGATE, ST32430N SUN2.1G, 0444> SCSI2 0/direct fixed
   38:       sd0: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors
   39:     
   40:     Disk #2
   41:       probe(esp0:1:0): max sync rate 10.00MB/s
   42:       sd1 at scsibus0 target 1 lun 0: <SEAGATE, ST32430N SUN2.1G, 0444> SCSI2 0/direct fixed
   43:       sd1: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors
   44:     
   45:     Disk #3
   46:       probe(esp0:2:0): max sync rate 10.00MB/s
   47:       sd2 at scsibus0 target 2 lun 0: <SEAGATE, ST11200N SUN1.05, 9500> SCSI2 0/direct fixed
   48:       sd2: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors
   49:     
   50:     Disk #4
   51:       probe(esp0:3:0): max sync rate 10.00MB/s
   52:       sd3 at scsibus0 target 3 lun 0: <SEAGATE, ST11200N SUN1.05, 8808 > SCSI2 0
   53:       sd3: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors
   54: 
   55: ## Configure Kernel Support
   56: 
   57: The following kernel configuration directive is needed to provide CCD device
   58: support. It is enabled in the GENERIC kernel:
   59: 
   60:     pseudo-device  ccd  4    # concatenated disk devices
   61: 
   62: In my kernel config, I also hard code SCSI ID associations to `/dev` device
   63: entries to prevent bad things from happening:
   64: 
   65:     sd0     at scsibus0 target 0 lun ?
   66:     # SCSI disk drives
   67:     sd1     at scsibus0 target 1 lun ?
   68:     # SCSI disk drives
   69:     sd2     at scsibus0 target 2 lun ?
   70:     # SCSI disk drives
   71:     sd3     at scsibus0 target 3 lun ?
   72:     # SCSI disk drives
   73:     sd4     at scsibus0 target 4 lun ?
   74:     # SCSI disk drives
   75:     sd5     at scsibus0 target 5 lun ?
   76:     # SCSI disk drives
   77:     sd6     at scsibus0 target 6 lun ?
   78:     # SCSI disk drives
   79: 
   80: ## Disklabel each volume member of the CCD
   81: 
   82: Each member disk of the CCD will need a special file system established. In this
   83: example, I will need to disklabel:
   84: 
   85:  * `/dev/rsd0c`
   86:  * `/dev/rsd1c`
   87:  * `/dev/rsd2c`
   88:  * `/dev/rsd3c`
   89: 
   90: *Note*: Always remember to disklabel the character device, not the block device,
   91: in `/dev/r{s,w}d*`.
   92: 
   93: *Note*: On all platforms, the `c` slice is symbolic of the entire NetBSD
   94: partition and is reserved.
   95: 
   96: You will probably want to remove any pre-existing disklabels on the disks in the
   97: CCD. This can be accomplished in one of two ways with the
   98: [[!template id=man name="dd" section="1"]] command:
   99: 
  100:     # dd if=/dev/zero of=/dev/rsd0c bs=8k count=1
  101:     # dd if=/dev/zero of=/dev/rsd1c bs=8k count=1
  102:     # dd if=/dev/zero of=/dev/rsd2c bs=8k count=1
  103:     # dd if=/dev/zero of=/dev/rsd3c bs=8k count=1
  104: 
  105: If your port uses a MBR (Master Boot Record) to partition the disks so that the
  106: NetBSD partitions are only part of the overall disk, and other OSs like Windows
  107: or Linux use other parts, you can void the MBR and all partitions on disk by
  108: using the command:
  109: 
  110:     # dd if=/dev/zero of=/dev/rsd0d bs=8k count=1
  111:     # dd if=/dev/zero of=/dev/rsd1d bs=8k count=1
  112:     # dd if=/dev/zero of=/dev/rsd2d bs=8k count=1
  113:     # dd if=/dev/zero of=/dev/rsd3d bs=8k count=1
  114: 
  115: This will make all data on the entire disk inaccessible. Note that the entire
  116: disk is slice `d` on i386 (and some other ports), and `c` elsewhere (e.g. on
  117: sparc). See the `kern.rawpartition` sysctl -- `3` means `d`, `2` means `c`.
  118: 
  119: The default disklabel for the disk will look similar to this:
  120: 
  121:     # disklabel -r sd0
  122:     [...snip...]
  123:     bytes/sector: 512
  124:     sectors/track: 116
  125:     tracks/cylinder: 9
  126:     sectors/cylinder: 1044
  127:     cylinders: 3992
  128:     total sectors: 4197405
  129:     [..snip...]
  130:     3 partitions:
  131:     #        size   offset    fstype   [fsize bsize   cpg]
  132:       c:  4197405       0     unused     1024  8192        # (Cyl.    0 - 4020*)
  133: 
  134: You will need to create one *slice* on the NetBSD partition of the disk that
  135: consumes the entire partition. The slice must begin at least one cylinder offset
  136: from the beginning of the disk/partition to provide space for the special CCD
  137: disklabel. The offset should be 1x sectors/cylinder (see following note).
  138: Therefore, the `size` value should be `total sectors` minus 1x
  139: `sectors/cylinder`. Edit your disklabel accordingly:
  140: 
  141:     # disklabel -e sd0
  142: 
  143: *Note*: The offset of a slice of type `ccd` must be a multiple of the
  144: `sectors/cylinder` value.
  145: 
  146: *Note*: Be sure to `export EDITOR=[path to your favorite editor]` before
  147: editing the disklabels.
  148: 
  149: *Note*: The slice must be fstype `ccd`.
  150: 
  151: Because there will only be one slice on this partition, you can recycle the `c`
  152: slice (normally reserved for symbolic uses). Change your disklabel to the
  153: following:
  154: 
  155:     3 partitions:
  156:     #        size   offset    fstype   [fsize bsize   cpg]
  157:       c:  4196361     1044       ccd                       # (Cyl. 1 - 4020*)
  158: 
  159: Optionally you can setup a slice other than `c` to use, simply adjust
  160: accordingly below:
  161: 
  162:     3 partitions:
  163:     #        size   offset    fstype   [fsize bsize   cpg]
  164:       a:  4196361     1044       ccd                       # (Cyl. 1 - 4020*)
  165:       c:  4197405       0     unused     1024  8192        # (Cyl. 0 - 4020*)
  166: 
  167: Be sure to write the label when you have completed. Disklabel will object to
  168: your disklabel and prompt you to re-edit if it does not pass its sanity checks.
  169: 
  170: ## Configure the CCD
  171: 
  172: Once all disks are properly labeled, you will need to generate a configuration
  173: file, `/etc/ccd.conf`. The file does not exist by default, and you will need to
  174: create a new one. The format is:
  175: 
  176:     #ccd    ileave    flags   component    devices
  177: 
  178: *Note*: For the `ileave`, if a value of zero is used then the disks are
  179: concatenated, but if you use a value equal to the `sectors/track` number the
  180: disks are interleaved.
  181: 
  182: Example in this case:
  183: 
  184:     # more /etc/ccd.conf
  185:     ccd0  0  none /dev/sd0c /dev/sd1c /dev/sd2c /dev/sd3c
  186: 
  187: *Note*: The CCD driver expects block device files as components. Be sure not to
  188: use character device files in the configuration.
  189: 
  190: ## Initialize the CCD device
  191: 
  192: Once you are confident that your CCD configuration is sane, you can initialize
  193: the device using the
  194: [[!template id=man name="ccdconfig" section="8"]]
  195: command: Configure:
  196: 
  197:     # ccdconfig -C -f /etc/ccd.conf
  198: 
  199: Unconfigure:
  200: 
  201:     # ccdconfig -u -f /etc/ccd.conf
  202: 
  203: Initializing the CCD device will activate `/dev` entries: `/dev/{,r}ccd#`:
  204: 
  205:     # ls -la  /dev/{,r}ccd0*
  206:     brw-r-----  1 root  operator   9, 0 Apr 28 21:35 /dev/ccd0a
  207:     brw-r-----  1 root  operator   9, 1 Apr 28 21:35 /dev/ccd0b
  208:     brw-r-----  1 root  operator   9, 2 May 12 00:10 /dev/ccd0c
  209:     brw-r-----  1 root  operator   9, 3 Apr 28 21:35 /dev/ccd0d
  210:     brw-r-----  1 root  operator   9, 4 Apr 28 21:35 /dev/ccd0e
  211:     brw-r-----  1 root  operator   9, 5 Apr 28 21:35 /dev/ccd0f
  212:     brw-r-----  1 root  operator   9, 6 Apr 28 21:35 /dev/ccd0g
  213:     brw-r-----  1 root  operator   9, 7 Apr 28 21:35 /dev/ccd0h
  214:     crw-r-----  1 root  operator  23, 0 Jun 12 20:40 /dev/rccd0a
  215:     crw-r-----  1 root  operator  23, 1 Apr 28 21:35 /dev/rccd0b
  216:     crw-r-----  1 root  operator  23, 2 Jun 12 20:58 /dev/rccd0c
  217:     crw-r-----  1 root  operator  23, 3 Apr 28 21:35 /dev/rccd0d
  218:     crw-r-----  1 root  operator  23, 4 Apr 28 21:35 /dev/rccd0e
  219:     crw-r-----  1 root  operator  23, 5 Apr 28 21:35 /dev/rccd0f
  220:     crw-r-----  1 root  operator  23, 6 Apr 28 21:35 /dev/rccd0g
  221:     crw-r-----  1 root  operator  23, 7 Apr 28 21:35 /dev/rccd0h
  222: 
  223: ## Create a 4.2BSD/UFS filesystem on the new CCD device
  224: 
  225: You may now disklabel the new virtual disk device associated with your CCD:
  226: 
  227:     # disklabel -e ccd0
  228: 
  229: Once again, there will be only one slice, so you may either recycle the `c`
  230: slice or create a separate slice for use.
  231: 
  232:     # disklabel -r ccd0
  233:     # /dev/rccd0c:
  234:     type: ccd
  235:     disk: ccd
  236:     label: default label
  237:     flags:
  238:     bytes/sector: 512
  239:     sectors/track: 2048
  240:     tracks/cylinder: 1
  241:     sectors/cylinder: 2048
  242:     cylinders: 6107
  243:     total sectors: 12508812
  244:     rpm: 3600
  245:     interleave: 1
  246:     trackskew: 0
  247:     cylinderskew: 0
  248:     headswitch: 0           # microseconds
  249:     track-to-track seek: 0  # microseconds
  250:     drivedata: 0
  251:     #        size   offset    fstype   [fsize bsize   cpg]
  252:       c: 12508812        0    4.2BSD     1024  8192    16  # (Cyl. 0 - 6107*)
  253: 
  254: The filesystem will then need to be formatted:
  255: 
  256:     # newfs /dev/rccd0c
  257:     Warning: 372 sector(s) in last cylinder unallocated
  258:     /dev/rccd0c:    12508812 sectors in 6108 cylinders of 1 tracks, 2048 sectors
  259:             6107.8MB in 382 cyl groups (16 c/g, 16.00MB/g, 3968 i/g)
  260:     
  261:     super-block backups (for fsck -b #) at:
  262:     [...]
  263: 
  264: ## Mount the filesystem
  265: 
  266: Once you have a created a file system on the CCD device, you can then mount the
  267: file system against a mount point on your system. Be sure to mount the slice
  268: labeled type `ffs` or `4.2BSD`:
  269: 
  270:     # mount /dev/ccd0c /mnt
  271: 
  272: Then:
  273: 
  274:     # export BLOCKSIZE=1024; df
  275:     Filesystem  1K-blocks     Used   Avail Capacity  Mounted on
  276:     /dev/sd6a      376155   320290   37057    89%    /
  277:     /dev/ccd0c    6058800       1  5755859     0%    /mnt
  278: 
  279: Congratulations, you now have a working CCD. To configure the CCD device at boot
  280: time, set `ccd=yes` in `/etc/rc.conf`. You can adjust `/etc/fstab` to get the
  281: filesystem mounted at boot:
  282: 
  283:     /dev/ccd0c  /home ffs     rw    1 2
  284: 

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb