High Availability (HA)

1 Introduction

The LoadMaster system can be deployed as a single unit or in an active/standby dual-unit configuration (HA). HA allows two physical or virtual machines to become one logical device. Only one of these units is active and handling traffic at any one time. One unit is active and the other is a hot standby (passive). This provides redundancy and resiliency, meaning if one LoadMaster goes down for any reason, the hot standby can become active, therefore avoiding any downtime.

HA in the LoadMaster for Azure and AWS products works a different way to that of a regular LoadMaster. For more information and instructions on how to configure HA on the LoadMaster for Azure or AWS, refer to the HA for Azure, Feature Description or HA for AWS, Feature Description document.

Three IP addresses are needed per active interface; one for the active unit, one for the passive unit and one IP address for the shared interface.

1.1 Document Purpose

The purpose of this document is to describe the HA feature in the KEMP LoadMaster and provide step-by-step instructions on how to configure HA.

It is also possible to cluster a number of LoadMasters. For further information, please refer to the Feature Description, LoadMaster Clustering.

1.2 Intended Audience

This document is intended to be read by anyone who is interested in learning about the HA feature in the KEMP LoadMaster.

2 Advantages of High Availability (HA)

Advantages of High Availability.png

The ultimate goal of redundant LoadMasters is to provide reliable traffic management, even if one LoadMaster becomes unavailable. The advantages of HA are as follows:

Eliminates a single point of failure

The second (passive) unit monitors the active unit to detect if a failure has occurred

Active connections and sessions are not lost when a fail-over occurs

 

3 Prerequisites

There are some prerequisites to be aware of before setting up HA:

Two LoadMasters must be set up and the appropriate licenses must be in place

LoadMasters must be located on the same subnet to be in a HA pair

LoadMasters must be in the same physical location

A layer 2 connection (Ethernet/VLAN) is required

The LoadMasters must not be located further than 100 meters from each other

Ensure that you have more than one interconnection between the two LoadMasters to avoid data loss or lack of availability

The two LoadMasters must use the same default gateway

Use Network Time Protocol (NTP) to keep times on LoadMasters up-to-date. This ensures that the times are correct on any logs and that Common Address Redundancy Protocol (CARP) message timestamps are in sync.

Ensure that any switches do not prevent MAC spoofing

Latency on the link between the two LoadMasters must be below 100 milliseconds

Multicast traffic flow is required in both directions between the devices

Three IP addresses are required for each subnet in which the LoadMaster is configured

4 HA Components

Two protocols are used by LoadMasters in an HA configuration to perform health checks and to synchronize the configuration between LoadMasters.

CARP:

Keeps updated on the health of the partner

Governs whether the LoadMaster will take the active role

The Use for HA Checks option enables CARP requests to be sent over enabled interfaces. This can be set for multiple interfaces.

When CARP is being used, packet analysis tools (such as Wireshark), incorrectly display the protocol used as Virtual Router Redundancy Protocol (VRRP). Any IP addresses displayed by the packet analysis tools are fictitious and are not part of the CARP protocol.

Sync:

Sync maintains a “single image view” of the LoadMaster settings. It keeps the LoadMaster up-to-date with changes made to Virtual Services and all other configurations.

Notable exceptions that are not synchronized are; time and passwords

Keeps the standby LoadMaster updated on persistence updates

5 Setting Up HA

5.1 Set up the First Unit

To build a HA LoadMaster environment there are a number of settings that must be carefully specified. Follow the steps below to set up HA:

1. Log in to the LoadMaster that is desired to be the active (master) unit.

2. In the main menu, select the System Configuration and click the HA option.

Set up the First Unit.png

3. If you have a clustering license, a screen will appear asking if you want to set up HA Mode or Clustering. To set up HA, select HA Mode and click Confirm.

Set up the First Unit_1.png

4. Select HA (First) Mode in the HA Mode drop-down.

Set up the First Unit_2.png

5. Click OK on the resulting message.

Do not reboot at this time.

Set up the First Unit_3.png

6. Click OK on the resulting message reminding not to forget to set the Shared IP address.

Ticking the Prevent this page from creating additional dialogs check box will stop any warning messages, such as this one, from appearing.

Set up the First Unit_4.png

7. Specify the desired shared IP address in the HA Shared IP address field and click Set Shared address.

8. A confirmation message may appear. Click OK.

Do not reboot or reconnect at this time.

9. Enter the IP address of the passive (slave) unit in the HA Partner IP address and click the Set Partner address button.

10. A confirmation message will appear. Click OK.

11. As of the 7.2.36 firmware, the LoadMaster selects a HA Virtual ID based on the shared IP address of the first configured interface (the last 8 bits). You can change the value to whatever you want (in the range 1 – 255) or you can keep it at the value it already selected.

Please ensure the virtual ID is unique on each LoadMaster on the network.

12. Configure any other settings as needed.

13. Click Reboot Now.

Set up the First Unit_5.png

14. Click Continue.

15. Refresh the page after the LoadMaster has rebooted (this may take a few minutes).

Set up the First Unit_6.png

A log in screen appears. After logging in, a different menu will appear than before. This is the Local Administration menu displayed for HA units. This menu has far fewer options. Only configuration settings pertaining to that specific unit are accessible using the Local Administration option. All management of the HA units should be done using the shared IP address. To see the full menu and configure the units, access the WUI of the shared IP address, which was specified above.

16. Log in to the shared IP WUI by entering the shared IP address in the address bar of the browser and pressing Enter.

In the top-left of the screen there are 2 indicator squares. Set up the First Unit_7.png These squares indicate the status of the HA pair. The left square always represents HA1 and the right represents HA2. The A represents which unit is active. The first or second HA unit can be opened by clicking the relevant status icon. Green and green status colors indicate a properly paired configuration. Currently, they will probably be green and red since the HA2 unit has not yet joined the pair. For an explanation of all icon colors and statuses, refer to the  HA Parameters section.

17. Go to HA Parameters in the main menu.

Set up the First Unit_8.png

18. Enter a different number (different from the IDs of other HA devices) in the HA Virtual ID text box and click Set Virtual ID. Using the same ID as other HA devices may cause problems.

All LoadMasters on the network that are or will be configured into HA pairs must be assigned unique HA Virtual ID numbers.

5.2 Set Up the Second Unit

Now that HA has been configured on the first unit, the second unit needs to be setup. Follow the steps below to do this.

1. Enter the IP address of the second unit in the address bar of the browser and press Enter.

Ensure to enter https:// before the IP address.

2. In the main menu, select System Configuration and click the HA option.

Set up the First Unit.png

3. If you have a clustering license, a screen will appear asking if you want to set up HA Mode or Clustering. To set up HA, select HA Mode and click Confirm.

Set Up the Second Unit.png

4. Select HA (Second) Mode as the HA Mode.

Set Up the Second Unit_1.png

5. Click OK on the resulting message.

Set Up the Second Unit_2.png

6. Click OK on the resulting message.

Ticking the Prevent this page from creating additional dialogs check box will stop any warning messages, such as this one, from appearing.

Set Up the Second Unit_3.png

7. Enter the HA Shared IP address and click the Set Shared address button.

The HA Shared IP address must be the same as the HA Shared IP address which was set when configuring the first unit in the Set up the First Unit section.

8. Click OK on the resulting message.

9. Click OK on the message asking to reconnect to the shared IP address.

10. Enter the IP address of the first (master) unit in the HA pair in the HA Partner IP address field and click Set Partner address.

11. Click OK on the resulting message.

12. Change any other settings as needed.

13. Click the Reboot Now button.

Set up the First Unit_5.png

14. Click Continue.

Passwords for the bal account are not synchronized across HA pairs, so ensure to use the same password on both units. Problems may occur if different passwords are used.

After rebooting, the HA pair will establish a TCP connection (using port 6973) between the 2 addresses. The synchronization process is started for the configuration.

The indicator squares should now show green and green. Set Up the Second Unit_4.png The A indicates the active unit of the pair. If the first synchronization attempt fails (that is, the icons are not green and green) a second attempt might be needed.

Set Up the Second Unit_5.png

On the home screen, the IP address field has changed. In addition to specifying the shared IP address of the pair, it also specifies the active unit in the pair. The left IP address is the shared address. The IP address in parentheses is the address of the current, active unit.

5.3 Enable the “Use for HA Checks” Option

Some guidelines relating to the Use for HA checks option are below:

If you have a physical LoadMaster, you can connect a direct cable on eth1 between both boxes. Leave the IP configuration blank. Select the Use for HA checks check box.

In a hardware configuration, if a direct cable has been deployed between both units over eth1 and there are IP addresses set in the interface management screen (and Use for HA checks is enabled on that interface) this causes problems because the LoadMaster will think it is a production link and if one of the LoadMasters reboot the other will fail too.

If the Use for HA checks check box is greyed out it means that this is the only interface configured to be used for HA checks and cannot be deselected.

To enable the Use for HA checks option, follow the steps below:

1. Go to the WUI of the shared IP address.

2. In the main menu, select System Configuration.

3. Select the relevant interface.

4. Select the Use for HA checks check box.

These steps can be repeated if the Use for HA checks option needs to be enabled on more than one interface.

5.4 Test Failover

Now that the HA units have been set up, failover can be tested if needed. The easiest way to do this is to reboot the active unit. To reboot the unit, follow the steps below:

1. Log in to the IP address of the active unit.

2. In the main menu, click Local Administration.

3. Select System Reboot.

Test Failover.png

4. Click Reboot.

Test Failover_1.png

5. A confirmation message may appear. Click OK.

Set up the First Unit_5.png

6. Click Continue.

When HA1 is back online, both HA status icons should be green. The A should have moved into the right green square. Test Failover_2.png This means that the secondary unit is now the active unit.

When using local certificates in HA mode – the shared IP inherits the local certificate from the master unit. So, if a slave unit has a different local certificate to the master and failover occurs, the shared IP inherits the local certificate of the slave (now master) unit.

6 How to Perform a Firmware Update on HA Pairs

KEMP recommends that firmware updates are performed outside of working hours. This is to ensure there is no interruption to client connectivity. If it has to be done during working hours we suggest that a window of maintenance is put in place.

Before updating the firmware, ensure that the Switch to Preferred Server drop-down list is set to No Preferred Host in System Configuration > HA Parameters.

To update the firmware on a HA pair; perform the following steps using the shared IP address:

1. Update the currently active LoadMaster (“A”).

2. When the update is complete, reboot A so that the passive unit becomes active.

3. When the first unit is back up, update the other unit.

4. When the update is complete, reboot other unit.

Now A should be active as it originally was.

7 HA Disaster Recovery Process (Replacing a Defective Unit)

In a disaster recovery scenario, you may need to replace one of the HA units. Prerequisites and installation steps for this scenario are provided in the sections below.

7.1 Prerequisites

There are some prerequisites that are needed before replacing a HA unit in a disaster recovery situation. These are listed below:

The new LoadMaster should be licensed:

- Verify that the license is valid for the relevant features, that is, the same features for which the other HA unit is licensed

Gather the following data:

- IP address of the LoadMaster

- IP address of the partner

- Shared IP address

7.2 Installation

Installation of a new HA unit should be performed during a maintenance window. Allow approximately 20 minutes for the installation.

Follow the steps below:

1. Ensure that the HA ports are connected and both machines can reach each other by using the Ping option in System Configuration > Logging Options > System Log Files > Debug Options.

2. Connect to the WUI on the local (machine) IP address of the new LoadMaster.

If any changes are needed to the network configuration to access the replacement unit (the WUI interface, Default Gateway, and so on), please verify that the settings are identical to the settings in the remaining active unit. If this cannot be guaranteed, it is advisable to continue with the approach as described in the Replacing HA Units section.

3. In the main menu of the WUI, select System Configuration and click the HA option.

Set up the First Unit.png

4. If you have a clustering license, a screen will appear asking if you want to set up HA Mode or Clustering. To set up HA, select HA Mode and click Confirm.

Installation.png

5. Select the relevant HA Mode (first or second).

6. Click OK.

7. Click OK again.

Installation_1.png

8. Enter the HA Shared address. Click Set Shared address.

9. Enter the HA Partner IP address. Click Set Partner address.

10. Click Reboot Now.

11. Wait 45 to 60 seconds, then click Continue.

12. Wait until the HA icons at the top of the screen are green.

13. If not yet done, connect the LoadMaster to the Real Server network and the open network side.

14. In the main menu, select Local Administration > System Reboot.

Test Failover.png

15. Click Reboot.

16. Click OK.

17. Click Continue.

It is optional, but KEMP recommends simulating a failover to the newly restored partner to check that everything is working OK. There are two ways to do this, both are detailed in the sections below.

7.2.1 Reboot the Active Unit

Rebooting the active unit should cause the new unit to take over. To do this, follow the steps below in the WUI of the active unit:

1. In the main menu of the WUI, select Local Administration > System Reboot.

Test Failover.png

2. Click Reboot.

3. Wait 45 to 60 seconds, then click Continue.

7.2.2 Change the Switch to Preferred Server Option

To change the Switch to Preferred Server option, follow the steps below in the WUI of the shared IP address:

1. In the main menu, select System Configuration > HA Parameters.

Change the Switch to Preferred.png

2. In the Switch to Preferred Server drop-down list, select the relevant option in order to set the new LoadMaster as the preferred master.

8 HA WUI Options

See below for descriptions of the various HA-related fields in the LoadMaster WUI.

8.1 HA and Clustering

Set up the First Unit.png

The HA section in the WUI is only called HA and Clustering if you have a LoadMaster license with clustering enabled. If you do not have clustering, or if you have already set up HA, this section is called HA Parameters and you will not see the screen shown above. If clustering has been configured, this section is called Cluster Control.

This screen describes both HA Mode and Clustering. Select the relevant option and click Confirm to continue.

If clustering is configured, the HA mode options become unavailable.

8.1.1 Interfaces

If the unit is part of an HA configuration, the following screen is displayed when one of the interfaces is clicked.

Interfaces.png

This screen tells the user:

The IP address of this LoadMaster

The HA shared IP address. This is the IP address used to configure the pair.

The IP address of the paired machine

Whether or not this interface is enabled for HA health-checking

The speed of the link (automatically detected)

Any alternate addresses on this interface

8.1.1.1 “Use for HA checks”

Some key points to note about this option are below:

The Use for HA checks check box must be selected on at least one interface that has connectivity from HA1 to HA2.

If the Use for HA checks check box is greyed out it means that this is the only interface configured to be used for HA checks and cannot be deselected.

This option should include at least one production interface, because if HA checks are only selected on non-production interfaces, the backup unit will not notice if a production interface goes down and will not take over for the incapacitated unit.

In a hardware configuration, if a direct cable is deployed between both units over eth1 and IP addresses have been set in the interface (and Use for HA Checks is enabled on that interface) this causes problems because the LoadMaster will think it is a production link and if one of the LoadMasters reboot, the other will fail too.

8.1.2  HA Parameters

 The role of the LoadMaster can be changed by setting the HA Mode.  If the HA Mode is set to HA (First) Mode or HA (Second) Mode, a prompt will appear reminding to add a shared IP. Changing the HA Mode will require a reboot, so after the details are set, click the Reboot button provided. Once the LoadMaster has rebooted, the HA Parameters menu option is available in the System Configuration section provided the role is not Non HA Mode. Configuring both units in the same HA Mode, for example, HA (First Mode) and HA (First) Mode, will result in severe operational problems as; not only will both units be master, both units will try to use the same IP address.

When logging in to the HA cluster, use the shared IP address to view and set the full functionality of the pair, apart from passwords and licensing. Logging in to the direct IP address of either one of the devices displays different menu options (see menus below).  Logging into one of the LoadMasters directly is usually reserved for maintenance.

HA Parameters.png

 

HA Parameters_1.png

 

When a LoadMaster is in HA mode, the following screen appears when the HA Parameters menu option is selected:

HA Parameters_2.png

HA Status

At the top of the screen, next to the time, icons denote the real-time status of the LoadMaster units in the cluster. There is an icon for each unit in the cluster. This status is maintained using an automatic ping between the units.

HA Parameters_3.png

Clicking on these icons opens the management interface of the relevant HA partner.

The four possible icons are: 

Green (with ‘A’)

HA Parameters_4.png

The unit is online and operational and the HA units are correctly paired.

The A in the middle of the square indicates that this is the master (active) unit.

Green (without ‘A’)

HA Parameters_5.png

The unit is online and operational and the HA units are correctly paired.

The absence of an ‘A’ in the middle of the square indicates that this is not the master unit (standby).

Red/Yellow

HA-red-sm

The unit is not operational. It may be offline or misconfigured. The unit is not ready to take over. It may be offline or incorrectly paired.

Blue

HA-blue-sm

When the unit reboots more than three times in 5 minutes it enters a pacified state. In this state the machine is only accessible using the direct machine WUI (not the shared WUI) and it is not participating in any HA activity. Therefore, no changes from the master are received and it will not take over if the master fails. To remove the unit from the pacified state, log in to the pacified LoadMaster through SSH or the console and reboot.

Grey

HA-grey-sm

The machine is in an indeterminate state and may require a reboot to return to operation. In some cases, this may mean both machines are active, that is, both are set to master, and something has gone seriously wrong. If rebooting does not solve the issue, call KEMP Support.

No HA icons

 

If the HA status squares are not appearing in the WUI, it probably means that HA is not enabled. Go to System Administration and select the HA option. Ensure the HA Mode is set to either First or Second.

In HA mode, each LoadMaster has its own IP address that is used only for diagnostic purposes directly on the unit.  The HA pair have a shared IP address over which the WUI is used to configure and manage the pair as a single entity.

   

There are a number of prerequisites that must be in place in order for HA to function correctly, refer to the Prerequisites section for a list of these prerequisites.

 HA Mode

If using a single LoadMaster, select Non HA Mode.   When setting up HA mode, one LoadMaster must be set to HA (First) Mode and the other HA (Second) Mode. HA will not operate if both units have the same HA Mode.

 HA Timeout

 CARP requests are sent every second from the master. The value selected in the HA Timeout drop-down list is the time that the master machine must be unavailable before a switchover occurs. With this option, the time it takes an HA cluster to detect a failure can be adjusted from 3 seconds to 15 seconds in 3 second increments. The default value is 9 seconds. A lower value will detect failures sooner, whereas a higher value gives better protection against a DOS attack.

 HA Initial Wait Time

How long after the initial boot of a LoadMaster, before the machine decides that it should become active. If the partner machine is running, then this value is ignored. This value can be changed to mitigate the time taken for some intelligent switches to detect that the LoadMaster has started and to bring up the link.

 HA Virtual ID

 When using multiple HA LoadMaster clusters (or other devices using CARP-like protocols) on the same network, this value uniquely identifies each cluster so that there are no potential unwanted interactions.

KEMP highly recommends using a higher value than 10, as any other HA device using the same ID could interfere with HA operations.

As of the 7.2.36 release, the LoadMaster selects a virtual ID based on the shared IP address of the first configured interface (the last 8 bits). It is selected and displayed once both the shared address and the partner address are set. You can change the value to whatever you want (in the range 1 – 255) or you can keep it at the value it already selected. Please ensure the virtual ID is unique on each LoadMaster on the network.

Switch to Preferred Server

 By default, neither partner in a HA cluster has priority. So that when a machine restarts after a switchover, the machine becomes the slave and stays in that state until forced to master. Specifying a preferred host means that when this machine restarts, it will always try to become master and the partner will revert to slave mode.

Some connections may be dropped during the switchover if a preferred host is specified.

For normal operating conditions, KEMP recommends selecting No Preferred Host.

 HA Update Interface

The interface used to synchronize the entire HA configuration within the HA cluster. Synchronization occurs every two minutes. The information is synchronized over SSH port 6973.

Force Partner Update

Immediately forces the configuration from the active to standby unit without waiting for a normal update.

 Inter HA L4 TCP Connection Updates

 When using L4 services, if updates are enabled it allows L4 connections to be maintained across a HA switchover. This option is ignored for L7 services.

 Inter HA L7 Persistency Updates

 When using L7 services, if this option is enabled it allows persistence information to be shared between the HA partners. If an HA failover occurs, the persistence information will not be lost. Enabling this option can have a significant performance impact.

HA Multicast Interface

The network interface used for multicast traffic which is used to synchronize Layer 4 and Layer 7 traffic when Inter HA Updates are enabled.

Use Virtual MAC Addresses

Selecting this option creates a shared MAC address for both units. When failover occurs, the LoadMaster will handle the MAC address handover too. This allows the switches to keep the same MAC address and not worry about ARP caches or stale records. This is useful when gratuitous ARPs (used in communicating changes in HA IP addresses to switches) are not allowed.

This option is not available in VLMs because they are not physically connected.

9 Troubleshooting

This section outlines troubleshooting steps for some common HA-related problems. If further help is needed, please contact KEMP Support.

9.1 General Troubleshooting Tips

General HA troubleshooting steps are below:

Check that the IP settings for the Interface, Partner and Shared IP address are correct. These settings can be found in System Configuration > Interfaces > eth0.

Log in to each of the single HA interface addresses and ensure the HA parameters are correct (Local Administration > HA Parameters):

- Ensure that unit 1 is set to HA (First) Mode and unit 2 is HA (Second) Mode

- Ensure that both units are on the same protocol and HA ID

Ensure that all of the IP addresses are available and are not in use by another device. IP conflict will cause numerous problems.

Shut down one or both LoadMasters and try to ping the IP address of each unit. If there is an answer, another device is using that IP address. Try the ‘arp -a’ or ‘netstat’ commands to find out more information on what device that is.

Set the HA Virtual ID (in Local Administration > HA Parameters) to something other than 1. The further up the range the better – avoid numbers from 1 to 10 as other HA devices may have those IDs and use Virtual Router Redundancy Protocol (VRRP).

The Virtual ID can conflict with any device on the network which is using VRRP. If there are multiple HA clusters on the same network, they must also have different Virtual IDs.

Check that the time of both units are in sync and if they are not, ensure that NTP is configured and running on both units.

Ensure there are no Virtual Services using TCP and port 6973 on the interface where synchronization is configured.

Ensure there are no Virtual Services on either of the HA individual addresses.

Ensure there are no Virtual Services using TCP and port 22 on a LoadMaster interface port.

9.2 No HA Status Squares are Visible in the WUI

If the HA status squares, for example No HA Status Squares are Visible.png, are not appearing in the WUI, it probably means that HA is not enabled. Go to System Administration > HA Parameters and ensure the HA Mode is set to either First or Second.

9.3 Green/Red HA Status Squares

If one of the HA status squares is red, check if one of the machines has crashed. If it has not crashed, try the following steps:

Confirm that multicast is allowed on the network

Confirm that IGMP snooping is disabled

Ensure that promiscuous mode and portfast is enabled

For VLMs; ensure that MAC spoofing is allowed

Reboot both of the units using a single IP

Ping between the units

Confirm the network settings

Check the HA settings on both machines

Select more than one interface for HA checks

For hardware LoadMasters;

- Connect eth1 with a direct cable, leave the interface unconfigured and select Use for HA checks

- Check that No Preferred Host is selected in the Switch to Preferred Server field. If HA works on eth1, the network could be the issue

- Connecting eth1 and only using this interface for HA checks could lead to problems if the production interface goes down because HA changeover might not occur

- Run a TCP dump on both units and the switch in between and confirm that VRRP signals are sent and received

Change the HA Virtual ID to something other than 1 (preferably higher than 10)

Increase the value of the HA timeouts

Move one or both VLMs so that they are on the same physical host

9.4 Blue HA Status Square

If there is a blue HA status square, follow the steps below:

1. Let the affected unit run for 10 minutes.

While waiting you can check the HA parameters to ensure they are configured correctly.

2. After that, shut down the blue unit for 5 minutes and restart using SSH or the console.

3. Confirm the network and HA settings.

9.5 Both Units are Active and the WUI is Unresponsive – Blue or Red Status Square

This usually means that both units are set to the same HA Mode, for example HA (First) Mode. This will cause both units to be master as well as try to take the same IP address. This causes serious problems with all functions of the LoadMaster.

9.6 Grey HA Status Square

If there is a grey HA status square, call KEMP Technical Support.

9.7 Virtual Services Temporarily Unavailable After Failover

If Virtual Services are temporarily unavailable after a failover, try flushing the ARP cache on the next-hop Layer 3 device to which the LoadMaster is connected.

If that does not work, activate Virtual MAC and flush the ARP cache. Activating the Virtual MAC requires a reboot.

The Virtual MAC option is not available in VLMs due to the inability to physically influence the units.

To activate Virtual MAC, follow the steps below:

1. In the shared IP address WUI, go to System Configuration > HA Parameters.

2. Enable the Use Virtual MAC Addresses option.

Selecting this option creates a shared MAC address for both units. When failover occurs, the LoadMaster will handle the MAC address handover too. This allows the switches to keep the same MAC address and not worry about ARP caches or stale records. This is useful when gratuitous ARPs (used in communicating changes in HA IP addresses to switches) are not allowed.

9.8 No Access to WUI on HA1/2 or Shared

If the WUI is inaccessible using any of the three IP addresses, try the steps below:

Wait 5 minutes and try again. Sometimes WUI access has not yet been activated, even though the LoadMaster is responding to pings.

Try a different web browser

Clear the web browser cache

Try from a different computer

Log in to the console (of the IP address where the problem is occurring) using SSH, go to option 3 Local Administration, 4 Web Address and s Immediately Stop Web Server Access, which will then turn into s Immediately Start Web Server Access. This should restart hanging WUI access.

9.9 Nothing Works

If everything in this troubleshooting section fails, try to shut down one of the LoadMasters. The remaining LoadMaster should take over. This can remain in HA mode or be set to single – whichever works. This will provide a working solution until a more permanent fix can be found.

9.10 Issue with Hyper-V and HA on a Pair of VLMs

If you are having problems with Hyper-V and HA on a pair of VLMs, the following Microsoft TechNet article may help you to fix the problem: http://technet.microsoft.com/en-us/magazine/ff458341.aspx

9.11 HA Issue on VMware

9.11.1 Both Units Think That They Are the Master Unit

Both Units Think That They.png


Both Units Think That They_1.png

In some situations, when using HA in a VMware environment, both units might think that they are the master unit and they may see the other unit as pacified (represented by a blue HA status square). This issue could be caused by the VMware Switch configuration. To resolve this issue, select the options as outlined below on the virtual switch.

Both Units Think That They_2.png

Ensure that MAC Address Changes and Forged Transmits are both selected. When they have been selected, reboot one of the LoadMasters.

Both Units Think That They_3.png

Both Units Think That They_3.png

The units should now be able to communicate properly. The correct state should now appear in the HA status icons.

9.11.2 Two Virtual LoadMasters on Different Hosts

Having two Virtual LoadMasters on different hosts can also pose problems on VMware.

Two Virtual LoadMasters on.png

To resolve these issues, select the Notify Switches check box in the NIC Teaming tab of the virtual switch.

9.12 Synchronization Issue After Unbonding/Bonding an Interface

Occasionally, after unbonding a bonded interface or bonding an unbonded interface – the master/standby units cannot communicate. To fix this issue, restart the master unit.

10 Replacing HA Units

When replacing HA units, it is important to follow the correct steps in sequence to ensure that there is little or no downtime for end users. The below example has two old units which are to be replaced with two new units.

The steps below are written assuming that the new units are powered on and available to be provisioned and that all cabling is in place.

Replacing HA Units.png

No preferred master set

Replacement units:

Replacing HA Units_1.png

 

Replacing HA Units.png

 

Replacing HA Units.png

 

Update the firmware on the old units if possible. For further information on how to upgrade the firmware of a HA pair, refer to the How to Perform a Firmware Update on HA Pairs section and the Software Migration - Version 6 to 7, Technical Note document.

1. On the WUI of the individual unit being replaced, in the main menu, go to Local Administration > Backup/Restore.

Replacing HA Units_2.png

2. Click Create Backup File.

3. Save the backup file.

4. In the WUI of unit 1 (A), in the main menu, go to System Configuration > HA Parameters.

Replacing HA Units_3.png

5. Select Prefer First HA in the Switch to Preferred Server drop-down list.

Specifying a preferred host means that when the machine restarts, the preferred unit always becomes master and the partner will revert to slave mode. This may cause connection loss.

6. Remove old unit 2 (B) from production.

As old unit 1 (A) is master, this should not affect production.

7. In the WUI of new unit 2 (D), in the main menu, go to System Configuration > System Administration > Backup/Restore.

Replacing HA Units_4.png

8. Click Choose File.

9. Browse to and select the backup file.

10. Select the configuration(s) that need to be restored.

This is the backup from old unit 2 (B).

11. Click Restore Configuration.

It is not possible to restore a single machine configuration onto a HA machine, or restore a HA configuration onto a single machine. A unit must be put into HA mode before a backup from a HA machine can be used to restore the LoadMaster Base Configuration.

It is not possible to restore a configuration containing ESP-enabled Virtual Services onto a machine which is not enabled for ESP.

12. Click Reboot Now.

13. Click Continue.

14. In the main menu, go to Local Administration > Interfaces.

15. Select the relevant interface.

Replacing HA Units_5.png

16. Check that the Interface Address (address[/prefix]), the HA Shared IP address and HA Partner IP address are correct before adding it to the production unit.

17. In the main menu, go to Local Administration > HA Parameters.

Replacing HA Units_6.png

18. Ensure that the HA Virtual ID is correct and unique.

19. When the new unit 2 (D) is connected, open the WUI of the shared IP address and in the main menu, go to System Configuration > HA Parameters.

Replacing HA Units_7.png

20. Click Force Update.

This will copy the configuration from the old unit 1 (A) (Master) to the new unit 2 (D) (standby). This will take approximately 15 seconds.

21. When the update has completed, in the main menu, go to System Configuration > HA Parameters.

Replacing HA Units_8.png

22. Select Prefer Second HA in the Switch to Preferred Server drop-down list.

This will drop all connections while the second unit takes over as master.

Replacing HA Units.png

Preferred master: unit 2

Replacing HA Units_1.png

 

Replacing HA Units.png

 

Replacing HA Units.png

 

23. Now, old unit 1 (A) can be removed from production.

As new unit 2 (D) is master, this should not affect production.

24. On the WUI of the new unit 1 (C), in the main menu, go to System Configuration > System Administration > Backup/Restore.

Replacing HA Units_9.png

25. Click Choose File.

26. Browse to and select the backup file.

27. Select the configuration(s) that need to be restored.

This is the backup configuration from old unit 1 (A).

28. Click Restore Configuration.

It is not possible to restore a single machine configuration onto a HA machine and vice versa.

It is not possible to restore a configuration containing ESP-enabled Virtual Services onto a machine which is not enabled for ESP.

29. Click Reboot Now.

30. Click Continue.

31. In the main menu, go to Local Administration > Interfaces.

32. Select the relevant interface.

Replacing HA Units_10.png

33. Check that the Interface Address (address[/prefix]), the HA Shared IP address and HA Partner IP address are correct before adding it to the production unit.

34. In the main menu, go to Local Administration > HA Parameters.

Replacing HA Units_11.png

35. Ensure that the HA Virtual ID is correct and unique.

36. When new unit 1 (C) is connected, open the WUI of the shared IP address and in the main menu, go to System Configuration > HA Parameters.

Replacing HA Units_12.png

37. Click Force Update.

38. This will take approximately 15 seconds.

39. When the update is complete, in the main menu, go to System Configuration > HA Parameters.

40. In the Switch to Preferred Server drop-down list, switch the preferred host to the other unit or select No Preferred Host.

Replacing HA Units.png

Preferred master set to none

Current setup – two units in production

Replacing HA Units_1.png

 

Replacing HA Units.png

 

Replacing HA Units.png

 

If you were replacing faulty units, ensure to send the old faulty units back to KEMP for testing.

References

Unless otherwise specified, the following documents can be found at http://kemptechnologies.com/documentation.

HA for Azure, Feature Description

HA for AWS, Feature Description

Software Migration - Version 6 to 7, Technical Note

Web User Interface (WUI), Configuration Guide

Feature Description, LoadMaster Clustering

Document History

 

Date

Change

Reason for Change

Version

Resp.

Sep 2014

Release updates

Updates for 7.1-20 release

1.8

LB

Feb 2015

Minor changes

Enhancements made

1.9

LB

May 2015

Minor change

Enhancement made

1.10

LB

Oct 2015

Release updates

Updates for 7.1-30 release

3.0

LB

Dec 2015

Minor change

Enhancement made

4.0

LB

Jan 2016

Minor change

Updated Copyright Notices

5.0

LB

Mar 2016

Minor change

Enhancement made

6.0

LB

July 2016

Release updates

Updates for 7.1.35 release

7.0

LB

Oct 2016

Release updates

Updates for 7.2.36 release

8.0

LB

Jan 2017

Minor change

Enhancement made

9.0

LB

Mar 2017 Release updates Updates for 7.2.38 release 10.0 LB

 

Was this article helpful?

1 out of 1 found this helpful

Comments