Annotation of wikisrc/guide/carp.mdwn, revision 1.3

1.3     ! jdf         1: **Contents**
        !             2: 
        !             3: [[!toc levels=3]]
        !             4: 
1.1       jdf         5: # Introduction to the Common Address Redundancy Protocol (CARP)
                      6: 
                      7: See [[below|guide/caro#license]] for the license of this chapter.
                      8: 
1.2       jdf         9: CARP is the *Common Address Redundancy Protocol*. Its primary purpose is to
                     10: allow multiple hosts on the same network segment to share an IP address. CARP is
1.1       jdf        11: a secure, free alternative to the
                     12: [Virtual Router Redundancy Protocol](http://www.ietf.org/rfc/rfc3768.txt)
                     13: and the
                     14: [Hot Standby Router Protocol](http://www.ietf.org/rfc/rfc2281.txt).
                     15: 
1.2       jdf        16: CARP works by allowing a group of hosts on the same network segment to share an
                     17: IP address. This group of hosts is referred to as a *redundancy group*. The
                     18: redundancy group is assigned an IP address that is shared amongst the group
                     19: members. Within the group, one host is designated the *master* and the rest as
                     20: *backups*. The master host is the one that currently *holds* the shared IP; it
                     21: responds to any traffic or ARP requests directed towards it. Each host may
1.1       jdf        22: belong to more than one redundancy group at a time.
                     23: 
1.2       jdf        24: One common use for CARP is to create a group of redundant firewalls. The virtual
                     25: IP that is assigned to the redundancy group is configured on client machines as
                     26: the default gateway. In the event that the master firewall suffers a failure or
                     27: is taken offline, the IP will move to one of the backup firewalls and service
1.1       jdf        28: will continue unaffected.
                     29: 
1.2       jdf        30: While highly redundant and fault-tolerant hardware minimizes the need for CARP,
                     31: it doesn't erase it. There is no hardware fault tolerance that is capable of
                     32: helping if someone knocks out a power cord, or if your system administrator
                     33: types reboot in the wrong window. CARP also makes it easier to make the patch
                     34: and reboot cycle transparent to users, and easier to test a software or hardware
1.1       jdf        35: upgrade -- if it doesn't work, you can fall back to your spare until fixed.
                     36: 
1.2       jdf        37: There are, however, situations in which CARP won't help. CARP's design does
                     38: require that the members of a group be on the same physical subnet with a static
                     39: IP address, although with the introduction of the carpdev directive, there is no
                     40: more need for IP addresses on the physical interfaces. Similarly, services that
                     41: require a constant connection to the server (such as SSH or IRC) will not be
                     42: transparently transferred to the other system -- though in this case, CARP can
                     43: help with minimizing downtime. CARP by itself does not synchronize data between
                     44: applications, for example, manually duplicating data between boxes with rsync,
1.1       jdf        45: or whatever is appropriate for your application.
                     46: 
                     47: CARP supports both IPv4 and IPv6.
                     48: 
                     49: ## CARP Operation
                     50: 
1.2       jdf        51: The master host in the group sends regular advertisements to the local network
                     52: so that the backup hosts know it's still alive. If the backup hosts don't hear
                     53: an advertisement from the master for a set period of time, then one of them will
                     54: take over the duties of master (whichever backup host has the lowest configured
                     55: advbase and advskew values). It is possible for multiple CARP groups to exist on
                     56: the same network segment. CARP advertisements contain the Virtual Host ID which
                     57: allows group members to identify which redundancy group the advertisement
1.1       jdf        58: belongs to.
                     59: 
1.2       jdf        60: In order to prevent a malicious user on the network segment from spoofing CARP
                     61: advertisements, each group can be configured with a password. Each CARP packet
1.1       jdf        62: sent to the group is then protected by an SHA1 HMAC.
                     63: 
                     64: ## Configuring CARP
                     65: 
1.2       jdf        66: Each redundancy group is represented by a
                     67: [carp(4)](http://netbsd.gw.com/cgi-bin/man-cgi?carp+4+NetBSD-5.0.1+i386) virtual
                     68: network interface. As such, CARP is configured using
                     69: [ifconfig(8)](http://netbsd.gw.com/cgi-bin/man-cgi?ifconfig+8+NetBSD-5.0.1+i386)
1.1       jdf        70: The follow options are available:
                     71: 
1.2       jdf        72:  * `carpN` -- The name of the carp virtual interface where N is an integer that
1.1       jdf        73:    represents the interface's number (e.g., carp 0).
                     74: 
1.2       jdf        75:  * `vhid` -- The Virtual Host ID. This is a unique number that is used to
                     76:    identify the redundancy group to other nodes on the network. Acceptable
                     77:    values are from 1 to 255. This allows for multiple redundancy groups to exist
1.1       jdf        78:    on the same network.
                     79: 
1.2       jdf        80:  * `password` -- The authentication password to use when talking to other
                     81:    CARP-enabled hosts in this redundancy group. This must be the same on all
1.1       jdf        82:    members of the redundancy group.
                     83: 
1.2       jdf        84:  * `carpdev` -- This optional parameter specifies the physical network interface
                     85:    that belongs to this redundancy group. By default, CARP will try to determine
                     86:    which interface to use by looking for a physical interface that is in the
                     87:    same subnet as the `ipaddress` and `mask` combination given to the carp
1.1       jdf        88:    interface.
                     89: 
1.2       jdf        90:  * `advbase` -- This optional parameter specifies how often, in seconds, to
                     91:    advertise that we're a member of the redundancy group. The default is 1
1.1       jdf        92:    second. Acceptable values are from 1 to 255.
                     93: 
1.2       jdf        94:  * `advskew` -- This optional parameter specifies how much to skew the advbase
                     95:    when sending CARP advertisements. By manipulating advbase, the master CARP
                     96:    host can be chosen. The higher the number, the less preferred the host will
                     97:    be when choosing a master. The default is 0. Acceptable values are from 1
                     98:    to 254.
1.1       jdf        99: 
1.2       jdf       100:  * `state` -- Force a carp interface into a certain state. Valid bits are
1.1       jdf       101:    `init`, `backup` and `master`.
                    102: 
1.2       jdf       103:  * `ipaddress` -- This is the shared IP address assigned to the redundancy
                    104:    group. This address does not have to be in the same subnet as the IP address
                    105:    on the physical interface (if present). This address needs to be the same on
1.1       jdf       106:    all hosts in the group, however.
                    107: 
                    108:  * `mask` -- The subnet mask of the shared IP.
                    109: 
1.2       jdf       110: Further CARP behaviour can be controlled via
1.1       jdf       111: [sysctl(8)](http://netbsd.gw.com/cgi-bin/man-cgi?sysctl+8+NetBSD-5.0.1+i386):
                    112: 
1.2       jdf       113:  * `net.inet.carp.allow` -- Accept incoming CARP packets or not. Default is 1
1.1       jdf       114:    (yes).
                    115: 
1.2       jdf       116:  * `net.inet.carp.preempt` -- Allow hosts within a redundancy group that have a
                    117:    better advbase and advskew to preempt the master. In addition, this option
                    118:    also enables failing over all interfaces in the event that one interface goes
                    119:    down. If one physical CARP-enabled interface goes down, CARP will change
                    120:    advskew to 240 on all other CARP-enabled interfaces, in essence, failing
1.1       jdf       121:    itself over. This option is 0 (disabled) by default.
                    122: 
                    123:  * `net.inet.carp.log` -- Log bad CARP packets. Default is 0 (disabled).
                    124: 
1.2       jdf       125:  * `net.inet.carp.arpbalance` -- Load balance traffic across multiple redundancy
1.1       jdf       126:    group hosts. Default is 0 (disabled). See
1.2       jdf       127:    [carp(4)](http://netbsd.gw.com/cgi-bin/man-cgi?carp+4+NetBSD-5.0.1+i386) for
1.1       jdf       128:    more information.
                    129: 
                    130: ## Enabling CARP Support
                    131: 
                    132: CARP support is not enabled by default.
                    133: 
                    134: To use
                    135: [carp(4)](http://netbsd.gw.com/cgi-bin/man-cgi?carp+4+NetBSD-5.0.1+i386),
1.2       jdf       136: you need a kernel with support for the `carp` pseudo-device. Make sure the
1.1       jdf       137: following line is in your kernel configuration file:
                    138: 
                    139:     pseudo-device   carp        # CARP
                    140: 
1.2       jdf       141: After configuring the `carp` pseudo-device in your kernel configuration, you
1.1       jdf       142: must recompile your kernel and reboot to enable carp support.
                    143: 
                    144: ## CARP Example
                    145: 
                    146: An example CARP configuration:
                    147: 
                    148:     # sysctl -w net.inet.carp.allow=1
                    149:     # ifconfig carp0 create
                    150:     # ifconfig carp0 vhid 1 pass lanpasswd \
                    151:          carpdev em0 advskew 100 10.0.0.1 255.255.255.0
                    152: 
                    153: This sets up the following:
                    154: 
                    155:  * Enables receipt of CARP packets (this is the default setting)
                    156: 
                    157:  * Creates an carp interface.
                    158: 
1.2       jdf       159:  * Configures carp0 for virtual host `#1`, enables a password (`lanpasswd`), sets
                    160:    `em0` as the interface belonging to the group, and makes this host a backup due
                    161:    to the advskew of 100 (assuming of course that the master is set up with an
                    162:    advskew less than 100). The shared IP assigned to this group is
1.1       jdf       163:    10.0.0.1/255.255.255.0.
                    164: 
                    165: Running ifconfig on carp0 shows the status of the interface:
                    166: 
                    167:     # ifconfig carp0
                    168:     carp0: flags=8802<UP,BROADCAST,SIMPLEX,MULTICAST> mtu 1500
                    169:          carp: BACKUP carpdev em0 vhid 1 advbase 1 advskew 100
                    170:          inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
                    171: 
                    172: ## Advanced CARP configuration
                    173: 
1.2       jdf       174: The following example creates a cluster of two highly-available, redundant
1.1       jdf       175: firewalls. The following diagram presents what we're trying to achieve:
                    176: 
1.2       jdf       177:              +----| WAN/Internet |----+
1.1       jdf       178:              |                        |
1.2       jdf       179:           em1|                        |em1
1.1       jdf       180:           +-----+                  +-----+
                    181:           | fw1 |                  | fw2 |
                    182:           +-----+                  +-----+
                    183:           em0|                        |em0
1.2       jdf       184:              |                        |
1.1       jdf       185:           ---+-------Shared LAN-------+---
                    186: 
1.2       jdf       187: Both firewalls are connected to the LAN on em0 and to a WAN/Internet connection
1.1       jdf       188: on em1. IP addresses are as follows:
                    189: 
                    190:  * Firewall 1 (fw1) em0: 172.16.0.1
                    191:  * Firewall 1 (fw1) em1: 192.0.2.1
                    192:  * Firewall 2 (fw2) em0: 172.16.0.2
                    193:  * Firewall 2 (fw2) em1: 192.0.2.2
                    194: 
                    195: The IP addresses we wish to share between the redundancy groups:
                    196: 
                    197:  * WAN/Internet Shared IP: 192.0.2.100
                    198:  * LAN Shared IP: 172.16.0.100
                    199: 
                    200: The network policy is that Firewall 1 (fw1) will be the preferred master.
                    201: 
                    202: The following configuration is for Firewall 1 (fw1):
                    203: 
                    204:     #Enable preemption and group interface failover
                    205:     # sysctl -w net.inet.carp.preempt=1
1.2       jdf       206:     
1.1       jdf       207:     #Configure CARP on the LAN side
                    208:     # ifconfig carp0 create
                    209:     # ifconfig carp0 vhid 1 pass lanpasswd carpdev em0 \
                    210:          172.16.0.100 255.255.255.0
1.2       jdf       211:     
1.1       jdf       212:     #Configure CARP on the WAN side
                    213:     # ifconfig carp1 create
                    214:     # ifconfig carp1 vhid 2 pass wanpasswd carpdev em1 \
                    215:          192.0.2.100 255.255.255.0
                    216: 
1.2       jdf       217: As mentioned before, our policy is for Firewall 1 to be the preferred master.
                    218: When configuring Firewall 2 we make the `advskew` a higher value since it's less
1.1       jdf       219: preferred to be the master.
                    220: 
                    221: The following configuration is for Firewall 2 (fw2):
                    222: 
                    223:     #Enable preemption and group interface failover
                    224:     # sysctl -w net.inet.carp.preempt=1
1.2       jdf       225:     
1.1       jdf       226:     #Configure CARP on the LAN side
                    227:     # ifconfig carp0 create
                    228:     # ifconfig carp0 vhid 1 pass lanpasswd carpdev em0 \
                    229:          advskew 128 172.16.0.100 255.255.255.0
1.2       jdf       230:     
1.1       jdf       231:     #Configure CARP on the WAN side
                    232:     # ifconfig carp1 create
                    233:     # ifconfig carp1 vhid 2 pass wanpasswd carpdev em1 \
                    234:          advskew 128 192.0.2.100 255.255.255.0
                    235: 
                    236: ## Forcing Failover of the Master
                    237: 
1.2       jdf       238: There can be times when it is necessary to failover or demote the master node on
                    239: purpose. Examples include taking the master node down for maintenance or when
                    240: troubleshooting a problem. The objective here is to gracefully fail over traffic
1.1       jdf       241: to one of the backup hosts so that users do not notice any impact.
                    242: 
1.2       jdf       243: To failover, shut down the carp interface on the master node. This will cause
                    244: the master to advertise itself with an *infinite* advbase and advskew. The
1.1       jdf       245: backup host(s) will see this and immediately take over the role of master.
                    246: 
                    247:     # ifconfig carp0 down
                    248: 
                    249: ## License
                    250: 
                    251: Copyright (c) 2005 Joel Knight <enabled@myrealbox.com>  
                    252: Permission to use, copy, modify, and distribute this documentation for  
                    253: any purpose with or without fee is hereby granted, provided that the  
                    254: above copyright notice and this permission notice appear in all copies.  
                    255: THE DOCUMENTATION IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL  
                    256: WARRANTIES WITH REGARD TO THIS DOCUMENTATION INCLUDING ALL IMPLIED  
                    257: WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE  
                    258: AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL  
                    259: DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR  
                    260: PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER  
                    261: TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR  
                    262: PERFORMANCE OF THIS DOCUMENTATION

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb