By using the Common Address Redundancy Protocol (CARP) on your FreeBSD boxes you can ensure that a certain virtual IP address is available as long as one machine is up at least. CARP forms an active/passive cluster, where one node is the master, which handles all the load. All other nodes are called slaves, which wait for the master to go down and to elect a new master. Additionally, you may setup CARP to form an active/active cluster, where the load is shared among all hosts. The source mac address is used to compute the responsible node for each ARP request. This is useful within your local network. If packets come through a router, then the cluster sees just the source mac address of that router. So one node handles all the load again.
My patch extends CARP to do loadbalancing on an IP basis. For each
incoming packet the responsible node is determined by (source ip address)
modulo (number of cluster nodes). Each node sends a helo packet using the
same multicast framework as CARP itself. So all nodes know how many peers are
alive. TCP packets are handled if the local node has an established state or
if SYN and no ACK flags are set. Two new sysctls, net.inet.carp.ip_balance and
net.inet.carp.ip_balance_mask, are added to switch on and off IP loadbalancing
and to give a number how many leading bits of each source ip address should be
evaluated, respectively.
Don't forget to add device carp to your kernel config.