Teaming: Broadcom NetXtreme® 57XX User Guide
Load Balancing and Fault Tolerance
NOTE: See Broadcom Gigabit Ethernet Teaming Services for detailed information on the following topics:
- Glossary of Terms and Acronyms
- Teaming Concepts
- Software Components
- Hardware Requirements
- Supported Teaming by Processor
- Configuring Teaming by Operating System
- Supported Features by Team Type
- Selecting a Team Type
- Teaming Mechanisms
- Architecture
- Types of Teams
- Driver Support by Operating System
- Supported Teaming Speeds
- Teaming and Other Advanced Networking Features
- General Network Considerations
- Application Considerations
- Troubleshooting Teaming Problems
- Frequently-Asked Questions
- Event Log Messages
Overview
Adapter teaming allows you to group network adapters together to function as a team. The benefits of teaming include allowing membership to VLANs, providing load balancing between adapters, and offering fault tolerance. These benefits can be combined such that you can couple the functionality of load balancing for the load balance members and the capability of employing a failover with having the team participate on different VLANs.
Broadcom Advanced Server Program (BASP) is the Broadcom teaming software for Windows 2000, Windows Server 2003, and NetWare operating systems. For Windows operating systems, BASP is configured through the Broadcom Advanced Control Suite 2 (BACS2) utility. For NetWare operating systems, teams are configured by loading BASP with all the necessary frames for the team (see Configuring Teaming). For Linux operating systems, teaming is done with channel bonding (see Teaming with Channel Bonding).
BASP provides heterogeneous support for adapter teaming to include all Broadcom NetXtreme Gigabit Ethernet adapters.
BASP supports four types of load balancing teams:
Load Balancing and Fault Tolerance
Teaming provides traffic load balancing and fault tolerance (redundant adapter operation in the event that a network connection fails). When multiple adapters are installed in the same system, they can be grouped with up to four teams.
Each team can consist of up to eight adapters, with one adapter used as a standby for Smart Load Balancing and Failover (SLB) or SLB (Auto-Fallback Disabled) team types. If traffic is not identified on any of the adapter team member connections due to failure of the adapter, cable, or switch, the load will be distributed to the remaining team members with an active connection. In the event that all primary adapters fail, traffic will be distributed to the standby adapter. Existing sessions are maintained with no impact on the user.
Types of Teams
The available types of teams for the supported operating systems are shown in the following table:
Table 1. Types of Teams Operating System Available Types of Teams Windows 2000/
Windows Server 2003 Smart Load Balancing and FailoverLink Aggregation (802.3ad)Generic Trunking (FEC/GEC)/802.3ad-Draft StaticSLB (Auto-Fallback Disable) Linux Team adapters using the bonding kernel module and a channel bonding interface: Information on channel bonding can be found at http://www.redhat.com/docs/manuals/enterprise/RHEL-3-Manual/ref-guide/s1-modules-ethernet.html. NetWare Smart Load BalancingLink Aggregation (802.3ad)Generic Trunking
Smart Load Balancing™ and Failover
Smart Load Balancing™ and Failover is the Broadcom implementation of load balancing based on IP flow. This feature supports balancing IP traffic across multiple adapters (team members) in a bidirectional manner. This type of team provides automatic fault detection and dynamic failover to other team member or to a hot standby member. This is done independently of layer 3 protocol (IP, IPX, NetBEUI); rather, it works with existing layer 2 and layer 3 switches. No switch configuration (such as trunk, link aggregation) is necessary for this type of team to work.
NOTES:
- If you do not enable LiveLink™ when configuring SLB teams, disabling Spanning Tree Protocol (STP) at the switch or port is recommended. This minimizes the downtime due to spanning tree loop determination when failing over. LiveLink mitigates such issues.
- IPX balances only on the transmit side of the team; other protocols are limited to the primary adapter.
- If a team member is linked at 1000 Mbit/s and another team member is linked at 100 Mbit/s, most of the traffic is handled by the 1000 Mbit/s team member.
Link Aggregation (802.3ad)
This mode supports link aggregation and conforms to the IEEE 802.3ad (LACP) specification. Configuration software allows you to dynamically configure which adapters you want to participate in a given team. If the link partner is not correctly configured for 802.3ad link configuration, errors are detected and noted. With this mode, all adapters in the team are configured to receive packets for the same MAC address. The outbound load-balancing scheme is determined by our BASP driver. The team link partner determines the load-balancing scheme for inbound packets. In this mode, at least one of the link partners must be in active mode.
Generic Trunking (FEC/GEC)/802.3ad-Draft Static
The Generic Trunking (FEC/GEC)/802.3ad-Draft Static type of team is very similar to the Link Aggregation (802.3ad) type of team in that all adapters in the team are configured to receive packets for the same MAC address. The Generic Trunking (FEC/GEC)/802.3ad-Draft Static) type of team, however, does not provide LACP or marker protocol support. This type of team supports a variety of environments in which the adapter link partners are statically configured to support a proprietary trunking mechanism. For instance, this type of team could be used to support Lucent's OpenTrunk or Cisco's Fast EtherChannel (FEC). Basically, this type of team is a light version of the Link Aggregation (802.3ad) type of team. This approach is much simpler, in that there is not a formalized link aggregation control protocol (LACP). As with the other types of teams, the creation of teams and the allocation of physical adapters to various teams is done statically through user configuration software.
The Generic Trunking (FEC/GEC/802.3ad-Draft Static) type of team supports load balancing and failover for both outbound and inbound traffic.
SLB (Auto-Fallback Disable)
The SLB (Auto-Fallback Disable) type of team is identical to the Smart Load Balance and Failover type of team, with the following exceptionwhen the standby member is active, if a primary member comes back on line, the team continues using the standby member, rather than switching back to the primary member.
If any primary adapter assigned to a team is disabled, the team functions as a Smart Load Balancing and Failover type of team in which auto-fallback occurs.
All primary interfaces in a team participate in load-balancing operations by sending and receiving a portion of the total traffic. Standby interfaces take over in the event that all primary interfaces have lost their links.
Failover teaming provides redundant adapter operation (fault tolerance) in the event that a network connection fails. If the primary adapter in a team is disconnected because of failure of the adapter, cable, or switch port, the secondary team member becomes active, redirecting both inbound and outbound traffic originally assigned to the primary adapter. Sessions will be maintained, causing no impact to the user.
Limitations of Smart Load Balance and Failover/SLB (Auto-Fallback Disable) Types of Teams
Smart Load Balancing™ (SLB) is a protocol-specific scheme. The level of support for IP, IPX, and NetBEUI protocols is listed below.
Drivers for third-party network adapters must be NetWare Event Service Layer (NESL) compliant for NetWare to be fault-tolerant and load-balanced in a multivendor team.
The Smart Load Balancing type of team works with all Ethernet switches without having to configure the switch ports to any special trunking mode. Only IP traffic is load-balanced in both inbound and outbound directions. IPX traffic is load-balanced in the outbound direction only. Other protocol packets are sent and received through one primary interface only. Failover for non-IP traffic is supported only for Broadcom network adapters. The Generic Trunking type of team requires the Ethernet switch to support some form of port trunking mode (for example, Cisco's Gigabit EtherChannel or other switch vendor's Link Aggregation mode). The Generic Trunking type of team is protocol-independent, and all traffic should be load-balanced and fault-tolerant.
NOTE: If you do not enable LiveLink™ when configuring teams, disabling Spanning Tree Protocol (STP) at the switch is recommended. This minimizes the downtime due to the spanning tree loop determination when failing over. LiveLink mitigates such issues.
LiveLink™ Functionality
LiveLink™ functionality is a feature of BASP that is available only for the Smart Load Balancing™ and Failover type of teaming. The purpose of LiveLink is to detect network connectivity beyond the switch and to route traffic only through team members that have a live link. This function is accomplished though the teaming software (see ). The teaming software periodically probes (issues a link packet from each team member) one or more specified target network adapter(s). The probe target(s) responds when it receives the link packet. If a team member does not detect a response within a specified amount of time after a specified number of retries, the teaming software discontinues passing traffic through that team member. Later, if that team member begins to detect a response from a probe target, this indicates that the link has been restored, and the teaming software automatically resumes passing traffic through that team member. LiveLink works only with TCP/IP.
LiveLink™ functionality is supported in both 32-bit and 64-bit Windows operating systems (except for Windows NT), but is not supported in NetWare systems. See the Channel Bonding documentation for similar functionality in Linux Channel Bonding (see http://www.redhat.com/docs/manuals/enterprise/RHEL-3-Manual/ref-guide/s1-modules-ethernet.html).
Teaming and Large Send Offload/Checksum Offload Support
Large Send Offload (LSO) and Checksum Offload are automatically enabled for a team only if LSO and Checksum Offload are supported and configured on all members of a team.