File:  [NetBSD Developer Wiki] / wikisrc / guide / carp.mdwn
Revision 1.3: download - view: text, annotated - select for diffs
Thu Mar 21 11:11:20 2013 UTC (5 years, 3 months ago) by jdf
Branches: MAIN
CVS tags: HEAD
Add TOCs to all chapters.


[[!toc levels=3]]

# Introduction to the Common Address Redundancy Protocol (CARP)

See [[below|guide/caro#license]] for the license of this chapter.

CARP is the *Common Address Redundancy Protocol*. Its primary purpose is to
allow multiple hosts on the same network segment to share an IP address. CARP is
a secure, free alternative to the
[Virtual Router Redundancy Protocol](
and the
[Hot Standby Router Protocol](

CARP works by allowing a group of hosts on the same network segment to share an
IP address. This group of hosts is referred to as a *redundancy group*. The
redundancy group is assigned an IP address that is shared amongst the group
members. Within the group, one host is designated the *master* and the rest as
*backups*. The master host is the one that currently *holds* the shared IP; it
responds to any traffic or ARP requests directed towards it. Each host may
belong to more than one redundancy group at a time.

One common use for CARP is to create a group of redundant firewalls. The virtual
IP that is assigned to the redundancy group is configured on client machines as
the default gateway. In the event that the master firewall suffers a failure or
is taken offline, the IP will move to one of the backup firewalls and service
will continue unaffected.

While highly redundant and fault-tolerant hardware minimizes the need for CARP,
it doesn't erase it. There is no hardware fault tolerance that is capable of
helping if someone knocks out a power cord, or if your system administrator
types reboot in the wrong window. CARP also makes it easier to make the patch
and reboot cycle transparent to users, and easier to test a software or hardware
upgrade -- if it doesn't work, you can fall back to your spare until fixed.

There are, however, situations in which CARP won't help. CARP's design does
require that the members of a group be on the same physical subnet with a static
IP address, although with the introduction of the carpdev directive, there is no
more need for IP addresses on the physical interfaces. Similarly, services that
require a constant connection to the server (such as SSH or IRC) will not be
transparently transferred to the other system -- though in this case, CARP can
help with minimizing downtime. CARP by itself does not synchronize data between
applications, for example, manually duplicating data between boxes with rsync,
or whatever is appropriate for your application.

CARP supports both IPv4 and IPv6.

## CARP Operation

The master host in the group sends regular advertisements to the local network
so that the backup hosts know it's still alive. If the backup hosts don't hear
an advertisement from the master for a set period of time, then one of them will
take over the duties of master (whichever backup host has the lowest configured
advbase and advskew values). It is possible for multiple CARP groups to exist on
the same network segment. CARP advertisements contain the Virtual Host ID which
allows group members to identify which redundancy group the advertisement
belongs to.

In order to prevent a malicious user on the network segment from spoofing CARP
advertisements, each group can be configured with a password. Each CARP packet
sent to the group is then protected by an SHA1 HMAC.

## Configuring CARP

Each redundancy group is represented by a
[carp(4)]( virtual
network interface. As such, CARP is configured using
The follow options are available:

 * `carpN` -- The name of the carp virtual interface where N is an integer that
   represents the interface's number (e.g., carp 0).

 * `vhid` -- The Virtual Host ID. This is a unique number that is used to
   identify the redundancy group to other nodes on the network. Acceptable
   values are from 1 to 255. This allows for multiple redundancy groups to exist
   on the same network.

 * `password` -- The authentication password to use when talking to other
   CARP-enabled hosts in this redundancy group. This must be the same on all
   members of the redundancy group.

 * `carpdev` -- This optional parameter specifies the physical network interface
   that belongs to this redundancy group. By default, CARP will try to determine
   which interface to use by looking for a physical interface that is in the
   same subnet as the `ipaddress` and `mask` combination given to the carp

 * `advbase` -- This optional parameter specifies how often, in seconds, to
   advertise that we're a member of the redundancy group. The default is 1
   second. Acceptable values are from 1 to 255.

 * `advskew` -- This optional parameter specifies how much to skew the advbase
   when sending CARP advertisements. By manipulating advbase, the master CARP
   host can be chosen. The higher the number, the less preferred the host will
   be when choosing a master. The default is 0. Acceptable values are from 1
   to 254.

 * `state` -- Force a carp interface into a certain state. Valid bits are
   `init`, `backup` and `master`.

 * `ipaddress` -- This is the shared IP address assigned to the redundancy
   group. This address does not have to be in the same subnet as the IP address
   on the physical interface (if present). This address needs to be the same on
   all hosts in the group, however.

 * `mask` -- The subnet mask of the shared IP.

Further CARP behaviour can be controlled via

 * `net.inet.carp.allow` -- Accept incoming CARP packets or not. Default is 1

 * `net.inet.carp.preempt` -- Allow hosts within a redundancy group that have a
   better advbase and advskew to preempt the master. In addition, this option
   also enables failing over all interfaces in the event that one interface goes
   down. If one physical CARP-enabled interface goes down, CARP will change
   advskew to 240 on all other CARP-enabled interfaces, in essence, failing
   itself over. This option is 0 (disabled) by default.

 * `net.inet.carp.log` -- Log bad CARP packets. Default is 0 (disabled).

 * `net.inet.carp.arpbalance` -- Load balance traffic across multiple redundancy
   group hosts. Default is 0 (disabled). See
   [carp(4)]( for
   more information.

## Enabling CARP Support

CARP support is not enabled by default.

To use
you need a kernel with support for the `carp` pseudo-device. Make sure the
following line is in your kernel configuration file:

    pseudo-device   carp        # CARP

After configuring the `carp` pseudo-device in your kernel configuration, you
must recompile your kernel and reboot to enable carp support.

## CARP Example

An example CARP configuration:

    # sysctl -w net.inet.carp.allow=1
    # ifconfig carp0 create
    # ifconfig carp0 vhid 1 pass lanpasswd \
         carpdev em0 advskew 100

This sets up the following:

 * Enables receipt of CARP packets (this is the default setting)

 * Creates an carp interface.

 * Configures carp0 for virtual host `#1`, enables a password (`lanpasswd`), sets
   `em0` as the interface belonging to the group, and makes this host a backup due
   to the advskew of 100 (assuming of course that the master is set up with an
   advskew less than 100). The shared IP assigned to this group is

Running ifconfig on carp0 shows the status of the interface:

    # ifconfig carp0
    carp0: flags=8802<UP,BROADCAST,SIMPLEX,MULTICAST> mtu 1500
         carp: BACKUP carpdev em0 vhid 1 advbase 1 advskew 100
         inet netmask 0xffffff00 broadcast

## Advanced CARP configuration

The following example creates a cluster of two highly-available, redundant
firewalls. The following diagram presents what we're trying to achieve:

             +----| WAN/Internet |----+
             |                        |
          em1|                        |em1
          +-----+                  +-----+
          | fw1 |                  | fw2 |
          +-----+                  +-----+
          em0|                        |em0
             |                        |
          ---+-------Shared LAN-------+---

Both firewalls are connected to the LAN on em0 and to a WAN/Internet connection
on em1. IP addresses are as follows:

 * Firewall 1 (fw1) em0:
 * Firewall 1 (fw1) em1:
 * Firewall 2 (fw2) em0:
 * Firewall 2 (fw2) em1:

The IP addresses we wish to share between the redundancy groups:

 * WAN/Internet Shared IP:
 * LAN Shared IP:

The network policy is that Firewall 1 (fw1) will be the preferred master.

The following configuration is for Firewall 1 (fw1):

    #Enable preemption and group interface failover
    # sysctl -w net.inet.carp.preempt=1
    #Configure CARP on the LAN side
    # ifconfig carp0 create
    # ifconfig carp0 vhid 1 pass lanpasswd carpdev em0 \
    #Configure CARP on the WAN side
    # ifconfig carp1 create
    # ifconfig carp1 vhid 2 pass wanpasswd carpdev em1 \

As mentioned before, our policy is for Firewall 1 to be the preferred master.
When configuring Firewall 2 we make the `advskew` a higher value since it's less
preferred to be the master.

The following configuration is for Firewall 2 (fw2):

    #Enable preemption and group interface failover
    # sysctl -w net.inet.carp.preempt=1
    #Configure CARP on the LAN side
    # ifconfig carp0 create
    # ifconfig carp0 vhid 1 pass lanpasswd carpdev em0 \
         advskew 128
    #Configure CARP on the WAN side
    # ifconfig carp1 create
    # ifconfig carp1 vhid 2 pass wanpasswd carpdev em1 \
         advskew 128

## Forcing Failover of the Master

There can be times when it is necessary to failover or demote the master node on
purpose. Examples include taking the master node down for maintenance or when
troubleshooting a problem. The objective here is to gracefully fail over traffic
to one of the backup hosts so that users do not notice any impact.

To failover, shut down the carp interface on the master node. This will cause
the master to advertise itself with an *infinite* advbase and advskew. The
backup host(s) will see this and immediately take over the role of master.

    # ifconfig carp0 down

## License

Copyright (c) 2005 Joel Knight <>  
Permission to use, copy, modify, and distribute this documentation for  
any purpose with or without fee is hereby granted, provided that the  
above copyright notice and this permission notice appear in all copies.  

CVSweb for NetBSD wikisrc <> software: FreeBSD-CVSweb