File:  [NetBSD Developer Wiki] / wikisrc / guide / carp.mdwn
Revision 1.1: download - view: text, annotated - select for diffs
Mon Mar 4 23:00:44 2013 UTC (13 months, 2 weeks ago) by jdf
Branches: MAIN
CVS tags: HEAD
 * fix typo in index_bare
 * move CARP chapter from guide to wiki

# Introduction to the Common Address Redundancy Protocol (CARP)

See [[below|guide/caro#license]] for the license of this chapter.

CARP is the *Common Address Redundancy Protocol*. Its primary purpose is to 
allow multiple hosts on the same network segment to share an IP address. CARP is 
a secure, free alternative to the
[Virtual Router Redundancy Protocol](http://www.ietf.org/rfc/rfc3768.txt)
and the
[Hot Standby Router Protocol](http://www.ietf.org/rfc/rfc2281.txt).

CARP works by allowing a group of hosts on the same network segment to share an 
IP address. This group of hosts is referred to as a *redundancy group*. The 
redundancy group is assigned an IP address that is shared amongst the group 
members. Within the group, one host is designated the *master* and the rest as 
*backups*. The master host is the one that currently *holds* the shared IP; it 
responds to any traffic or ARP requests directed towards it. Each host may 
belong to more than one redundancy group at a time.

One common use for CARP is to create a group of redundant firewalls. The virtual 
IP that is assigned to the redundancy group is configured on client machines as 
the default gateway. In the event that the master firewall suffers a failure or 
is taken offline, the IP will move to one of the backup firewalls and service 
will continue unaffected.

While highly redundant and fault-tolerant hardware minimizes the need for CARP, 
it doesn't erase it. There is no hardware fault tolerance that is capable of 
helping if someone knocks out a power cord, or if your system administrator 
types reboot in the wrong window. CARP also makes it easier to make the patch 
and reboot cycle transparent to users, and easier to test a software or hardware 
upgrade -- if it doesn't work, you can fall back to your spare until fixed.

There are, however, situations in which CARP won't help. CARP's design does 
require that the members of a group be on the same physical subnet with a static 
IP address, although with the introduction of the carpdev directive, there is no 
more need for IP addresses on the physical interfaces. Similarly, services that 
require a constant connection to the server (such as SSH or IRC) will not be 
transparently transferred to the other system -- though in this case, CARP can 
help with minimizing downtime. CARP by itself does not synchronize data between 
applications, for example, manually duplicating data between boxes with rsync, 
or whatever is appropriate for your application.

CARP supports both IPv4 and IPv6.

## CARP Operation

The master host in the group sends regular advertisements to the local network 
so that the backup hosts know it's still alive. If the backup hosts don't hear 
an advertisement from the master for a set period of time, then one of them will 
take over the duties of master (whichever backup host has the lowest configured 
advbase and advskew values). It is possible for multiple CARP groups to exist on 
the same network segment. CARP advertisements contain the Virtual Host ID which 
allows group members to identify which redundancy group the advertisement 
belongs to.

In order to prevent a malicious user on the network segment from spoofing CARP 
advertisements, each group can be configured with a password. Each CARP packet 
sent to the group is then protected by an SHA1 HMAC.

## Configuring CARP

Each redundancy group is represented by a 
[carp(4)](http://netbsd.gw.com/cgi-bin/man-cgi?carp+4+NetBSD-5.0.1+i386) virtual 
network interface. As such, CARP is configured using 
[ifconfig(8)](http://netbsd.gw.com/cgi-bin/man-cgi?ifconfig+8+NetBSD-5.0.1+i386) 
The follow options are available:

 * `carpN` -- The name of the carp virtual interface where N is an integer that 
   represents the interface's number (e.g., carp 0).

 * `vhid` -- The Virtual Host ID. This is a unique number that is used to 
   identify the redundancy group to other nodes on the network. Acceptable 
   values are from 1 to 255. This allows for multiple redundancy groups to exist 
   on the same network.

 * `password` -- The authentication password to use when talking to other 
   CARP-enabled hosts in this redundancy group. This must be the same on all 
   members of the redundancy group.

 * `carpdev` -- This optional parameter specifies the physical network interface 
   that belongs to this redundancy group. By default, CARP will try to determine 
   which interface to use by looking for a physical interface that is in the 
   same subnet as the `ipaddress` and `mask` combination given to the carp 
   interface.

 * `advbase` -- This optional parameter specifies how often, in seconds, to 
   advertise that we're a member of the redundancy group. The default is 1 
   second. Acceptable values are from 1 to 255.

 * `advskew` -- This optional parameter specifies how much to skew the advbase 
   when sending CARP advertisements. By manipulating advbase, the master CARP 
   host can be chosen. The higher the number, the less preferred the host will 
   be when choosing a master. The default is 0. Acceptable values are from 1 to 
   254.

 * `state` -- Force a carp interface into a certain state. Valid bits are 
   `init`, `backup` and `master`.

 * `ipaddress` -- This is the shared IP address assigned to the redundancy 
   group. This address does not have to be in the same subnet as the IP address 
   on the physical interface (if present). This address needs to be the same on 
   all hosts in the group, however.

 * `mask` -- The subnet mask of the shared IP.

Further CARP behaviour can be controlled via 
[sysctl(8)](http://netbsd.gw.com/cgi-bin/man-cgi?sysctl+8+NetBSD-5.0.1+i386):

 * `net.inet.carp.allow` -- Accept incoming CARP packets or not. Default is 1 
   (yes).

 * `net.inet.carp.preempt` -- Allow hosts within a redundancy group that have a 
   better advbase and advskew to preempt the master. In addition, this option 
   also enables failing over all interfaces in the event that one interface goes 
   down. If one physical CARP-enabled interface goes down, CARP will change 
   advskew to 240 on all other CARP-enabled interfaces, in essence, failing 
   itself over. This option is 0 (disabled) by default.

 * `net.inet.carp.log` -- Log bad CARP packets. Default is 0 (disabled).

 * `net.inet.carp.arpbalance` -- Load balance traffic across multiple redundancy 
   group hosts. Default is 0 (disabled). See
   [carp(4)](http://netbsd.gw.com/cgi-bin/man-cgi?carp+4+NetBSD-5.0.1+i386) for 
   more information.

## Enabling CARP Support

CARP support is not enabled by default.

To use
[carp(4)](http://netbsd.gw.com/cgi-bin/man-cgi?carp+4+NetBSD-5.0.1+i386),
you need a kernel with support for the `carp` pseudo-device. Make sure the 
following line is in your kernel configuration file:

    pseudo-device   carp        # CARP

After configuring the `carp` pseudo-device in your kernel configuration, you 
must recompile your kernel and reboot to enable carp support.

## CARP Example

An example CARP configuration:

    # sysctl -w net.inet.carp.allow=1
    # ifconfig carp0 create
    # ifconfig carp0 vhid 1 pass lanpasswd \
         carpdev em0 advskew 100 10.0.0.1 255.255.255.0

This sets up the following:

 * Enables receipt of CARP packets (this is the default setting)

 * Creates an carp interface.

 * Configures carp0 for virtual host `#1`, enables a password (`lanpasswd`), sets 
   `em0` as the interface belonging to the group, and makes this host a backup due 
   to the advskew of 100 (assuming of course that the master is set up with an 
   advskew less than 100). The shared IP assigned to this group is 
   10.0.0.1/255.255.255.0.

Running ifconfig on carp0 shows the status of the interface:

    # ifconfig carp0
    carp0: flags=8802<UP,BROADCAST,SIMPLEX,MULTICAST> mtu 1500
         carp: BACKUP carpdev em0 vhid 1 advbase 1 advskew 100
         inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255

## Advanced CARP configuration

The following example creates a cluster of two highly-available, redundant 
firewalls. The following diagram presents what we're trying to achieve:

             +----| WAN/Internet |----+ 
             |                        |
          em1|                        |em1   
          +-----+                  +-----+
          | fw1 |                  | fw2 |
          +-----+                  +-----+
          em0|                        |em0
             |                        | 
          ---+-------Shared LAN-------+---
        

Both firewalls are connected to the LAN on em0 and to a WAN/Internet connection 
on em1. IP addresses are as follows:

 * Firewall 1 (fw1) em0: 172.16.0.1
 * Firewall 1 (fw1) em1: 192.0.2.1
 * Firewall 2 (fw2) em0: 172.16.0.2
 * Firewall 2 (fw2) em1: 192.0.2.2

The IP addresses we wish to share between the redundancy groups:

 * WAN/Internet Shared IP: 192.0.2.100
 * LAN Shared IP: 172.16.0.100

The network policy is that Firewall 1 (fw1) will be the preferred master.

The following configuration is for Firewall 1 (fw1):

    #Enable preemption and group interface failover
    # sysctl -w net.inet.carp.preempt=1

    #Configure CARP on the LAN side
    # ifconfig carp0 create
    # ifconfig carp0 vhid 1 pass lanpasswd carpdev em0 \
         172.16.0.100 255.255.255.0

    #Configure CARP on the WAN side
    # ifconfig carp1 create
    # ifconfig carp1 vhid 2 pass wanpasswd carpdev em1 \
         192.0.2.100 255.255.255.0

As mentioned before, our policy is for Firewall 1 to be the preferred master. 
When configuring Firewall 2 we make the `advskew` a higher value since it's less 
preferred to be the master.

The following configuration is for Firewall 2 (fw2):

    #Enable preemption and group interface failover
    # sysctl -w net.inet.carp.preempt=1

    #Configure CARP on the LAN side
    # ifconfig carp0 create
    # ifconfig carp0 vhid 1 pass lanpasswd carpdev em0 \
         advskew 128 172.16.0.100 255.255.255.0

    #Configure CARP on the WAN side
    # ifconfig carp1 create
    # ifconfig carp1 vhid 2 pass wanpasswd carpdev em1 \
         advskew 128 192.0.2.100 255.255.255.0

## Forcing Failover of the Master

There can be times when it is necessary to failover or demote the master node on 
purpose. Examples include taking the master node down for maintenance or when 
troubleshooting a problem. The objective here is to gracefully fail over traffic 
to one of the backup hosts so that users do not notice any impact.

To failover, shut down the carp interface on the master node. This will cause 
the master to advertise itself with an *infinite* advbase and advskew. The 
backup host(s) will see this and immediately take over the role of master.

    # ifconfig carp0 down

## License

Copyright (c) 2005 Joel Knight <enabled@myrealbox.com>  
Permission to use, copy, modify, and distribute this documentation for  
any purpose with or without fee is hereby granted, provided that the  
above copyright notice and this permission notice appear in all copies.  
THE DOCUMENTATION IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL  
WARRANTIES WITH REGARD TO THIS DOCUMENTATION INCLUDING ALL IMPLIED  
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE  
AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL  
DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR  
PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER  
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR  
PERFORMANCE OF THIS DOCUMENTATION

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb