CenturyLink HA

1 Introduction

When deploying an application using CenturyLink Cloud, you may need to provide load balancing and other application delivery functions such as content switching, SSL acceleration, Web Application Firewall, and Intrusion Prevention System (IPS). When using a Kemp LoadMaster for CenturyLink, you can address both application delivery and High Availability (HA).

Some of the features and associated benefits of the Virtual LoadMaster (VLM) are listed in the table below.

Feature

Benefit

Application ubiquity

Regardless of where the applications are deployed (cloud, on premises, or in hybrid environments), the VLM can load balance them.

Hybrid enhancement

The VLM manages applications deployed in hybrid infrastructures on premises and in the CenturyLink Cloud.

Scalable

Highly available Application Delivery Controllers (ADCs), deployed on-demand to meet load requirements.

Resilient

VLM Global Balancing (GEO) load balancing supports application instances across multiple sites to accommodate growth and deliver additional resilience.

Kemp provides two different approaches for deploying LoadMasters with HA in CenturyLink Cloud: native HA and HA using GEO. The approach to use is determined by the CenturyLink data center that the LoadMasters are deployed in.

CenturyLink native HA is the first approach and it leverages the CenturyLink load balancer to route traffic to a healthy LoadMaster. The CenturyLink load balancer features required for this functionality are available in select data centers. These data centers are marked as advanced capabilities.

The CenturyLink HA using GEO (GSLB) approach leverages the Kemp LoadMaster GEO functionality. This functionality requires DNS delegation and configuration to route the active LoadMaster. This approach does not require the CenturyLink load balancer.

1.1 Document Purpose

This document is intended to explain the two different configurations of HA for the Kemp LoadMaster in CenturyLink Cloud.

It is also possible to configure the LoadMaster using Application Program Interface (API) commands. For further details, refer to the Interface Description documents on the Kemp documentation page: https://kemptechnologies.com/documentation.

1.2 Intended Audience

This document is intended to be read by anyone who is interested in deploying a HA pair of LoadMasters in CenturyLink Cloud.

2 Setting Up HA

2.1 Set up the First Unit

To build a HA LoadMaster environment there are several settings that you must carefully specify. Follow the steps below to set up HA:

1. Log in to the LoadMaster that you want to be the active (master) unit.

2. In the main menu, select System Configuration and click HA Parameters.

Set up the First Unit.png

3. A screen appears asking if you want to set up HA Mode or Clustering. To set up HA, select HA Mode and click Confirm.

For instructions on configuring clustering, refer to the LoadMaster Clustering Feature Description on the Kemp documentation page.

Set up the First Unit_1.png

4. Select HA (First) Mode in the HA Mode drop-down.

Set up the First Unit_2.png

5. Click OK.

Do not reboot at this time.

Set up the First Unit_3.png

6. Click OK on the resulting message reminding not to forget to set the Shared IP address.

Selecting the Prevent this page from creating additional dialogs check box stops any warning messages, such as this one, from appearing.

123.png

7. Specify the desired shared IP address in the HA Shared IP address field and click Set Shared address.

8. A confirmation message may appear. Click OK.

Do not reboot or reconnect at this time.

9. Enter the IP address of the standby unit in the HA Partner IP address field and click Set Partner address.

10. A confirmation message appears. Click OK.

11. As of the 7.2.36 firmware, the LoadMaster selects a HA Virtual ID based on the shared IP address of the first configured interface (the last 8 bits). You can change the value to whatever you want (in the range 10 – 255) or you can keep it at the value it already selected.

Ensure the Virtual ID is unique for each HA pair on the network. When using multiple HA LoadMaster clusters (or other devices using CARP-like protocols) on the same network, this value uniquely identifies each cluster so that there are no potential unwanted interactions.

12. Configure any other settings as needed.

13. Click Reboot Now.

Set up the First Unit_5.png

14. Click Continue.

15. Refresh the page after the LoadMaster has rebooted (this may take a few minutes).

Set up the First Unit_6.png

A log in screen appears. After logging in, a different menu appears than before. This is the Local Administration menu displayed for HA units. This menu has far fewer options. Only configuration settings pertaining to that specific unit are accessible using the Local Administration option. All management of the HA units should be done using the shared IP address. To see the full menu and configure the units, access the WUI of the shared IP address, which was specified above.

16. Log in to the shared IP Web User Interface (WUI) by entering the shared IP address in the address bar of the browser and pressing Enter.

In the top-right of the screen there are two indicator squares. Set up the First Unit_7.png These squares indicate the status of the HA pair. The left square always represents HA1 and the right represents HA2. The A represents which unit is active. The first or second HA unit can be opened by clicking the relevant status icon. Green and green status colors indicate a properly paired configuration. Currently, the icons are probably green and red since the HA2 unit has not yet joined the pair. For an explanation of all icon colors and statuses, refer to the  HA Parameters section.

17. Go to HA Parameters in the main menu.

18. Enter a different number (different from the IDs of other HA devices) in the HA Virtual ID text box and click Set Virtual ID. Using the same ID as other HA devices may cause problems.

All HA pairs on the network must be assigned unique HA Virtual ID numbers.

2.2 Set Up the Second Unit

Now that HA has been configured on the first unit, the second unit must be set up. Follow the steps below to do this:

1. Enter the IP address of the second unit in the address bar of the browser and press Enter.

Ensure to enter https:// before the IP address.

2. In the main menu, select System Configuration and click the HA option.

Set up the First Unit.png

3. A screen appears asking if you want to set up HA Mode or Clustering. To set up HA, select HA Mode and click Confirm.

Set Up the Second Unit.png

4. Select HA (Second) Mode as the HA Mode.

Set Up the Second Unit_1.png

5. Click OK.

Set Up the Second Unit_2.png

6. Click OK.

Ticking the Prevent this page from creating additional dialogs check box stops any warning messages, such as this one, from appearing.

124.png

7. Enter the HA Shared IP address and click Set Shared address.

The HA Shared IP address must be the same as the HA Shared IP address which was set when configuring the first unit in the Set up the First Unit section.

8. Click OK.

9. Click OK on the message asking to reconnect to the shared IP address.

10. Enter the IP address of the first (master) unit in the HA pair in the HA Partner IP address field and click Set Partner address.

11. Click OK.

12. Ensure the HA Virtual ID is the same as it is on the other unit.

If they are not the same, the pairing fails.

13. Change any other settings as needed.

14. Click Reboot Now.

Set up the First Unit_5.png

15. Click Continue.

Passwords for the bal account are not synchronized across HA pairs, so ensure to use the same password on both units. Problems may occur if different passwords are used.

After rebooting, the HA pair establishes a TCP connection (using port 6973) between the two addresses. The synchronization process is started for the configuration.

The indicator squares should now be green and green. Set Up the Second Unit_4.png The A indicates the active unit of the pair. If the first synchronization attempt fails (that is, the icons are not green and green) a second attempt might be needed.

136.png

On the home screen, the IP address field has changed. In addition to specifying the shared IP address of the pair, it also specifies the IP address of the unit. The left IP address is the shared address. The IP address in parentheses is the address of the current unit.

2.3 Enable the 'Use for HA Checks' Option

Some guidelines relating to the Use for HA checks option are below:

  • If the Use for HA checks check box is grayed out it means that this is the only interface configured to be used for HA checks and it cannot be deselected.

To enable the Use for HA checks option, follow the steps below:

1. Go to the WUI of the shared IP address.

2. In the main menu, select System Configuration.

3. Select the relevant interface.

4. Select the Use for HA checks check box.

These steps can be repeated if you need to enable the Use for HA checks option on more than one interface.

2.4 Test Failover

Now that the HA units have been set up, failover can be tested if needed. The easiest way to do this is to reboot the active unit. To reboot the unit, follow the steps below:

1. Log in to the IP address of the active unit.

2. In the main menu, click Local Administration.

3. Select System Reboot.

Test Failover.png

4. Click Reboot.

Test Failover_1.png

5. A confirmation message may appear. Click OK.

Set up the First Unit_5.png

6. Click Continue.

When HA1 is back online, both HA status icons should be green. The A should have moved into the right green square. Test Failover_2.png This means that the secondary unit is now the active unit.

When using local certificates in HA mode – the shared IP inherits the local certificate from the master unit. So, if a standby unit has a different local certificate to the master and failover occurs, the shared IP inherits the local certificate of the standby (now master) unit.

3 Performing a Firmware Update on HA Pairs

Kemp recommends performing firmware updates outside of working hours. This ensures there is no interruption to client connectivity. If it has to be done during working hours, Kemp recommend scheduling a maintenance window.

Before updating the firmware, ensure the Switch to Preferred Server drop-down list is set to No Preferred Host in System Configuration > HA Parameters.

Kemp recommends updating the passive unit first and then updating the active unit. This causes only a single failover and minimal downtime, and is the preferred option for most customers. While this procedure does leave the currently passive LoadMaster as the active LoadMaster going forward, this usually has no consequences in most customer environments. However, it is also possible to update the currently active unit, failover to the passive unit, update the passive unit and then failover to the originally active unit.

To update the firmware on a HA pair using the recommended method; perform the following steps using the shared IP address:

1. Update the passive LoadMaster first (we will refer to this LoadMaster as B).

2. When the update is complete, reboot B.

3. When unit B is back up, update the active unit (we will refer to this unit as A).

4. When the update is complete, reboot A. Now B becomes active.

5. Ensure B is handling traffic.

4 HA WUI Options

See below for descriptions of the various HA-related fields in the LoadMaster WUI.

4.1 Interfaces

If the unit is part of a HA configuration, the following screen displays when you click one of the interfaces.

125.png

This screen tells the user:

  • The IP address of this LoadMaster (10.35.47.10 in this example).
  • The HA shared IP address (10.35.47.30 in this example). This is the IP address used to configure the pair.
  • The IP address of the paired machine (10.35.47.12 in this example).
  • Whether or not this interface is enabled for HA health-checking.
  • The speed of the link (automatically detected). If the link is down, it is indicated here.
  • Any alternate addresses on this interface.

4.1.1 Use for HA checks

Some key points to note about this option are below:

  • The Use for HA checks check box must be selected on at least one interface that has connectivity from HA1 to HA2.
  • If the Use for HA checks check box is grayed out it means that this is the only interface configured to be used for HA checks and cannot be deselected.
  • This option should include at least one production interface, because if HA checks are only selected on non-production interfaces, the backup unit does not notice if a production interface goes down and does not take over for the incapacitated unit.

4.2  HA Parameters

 You can change the role of the LoadMaster by setting the HA Mode. If the HA Mode is set to HA (First) Mode or HA (Second) Mode, a prompt appears reminding to add a shared IP. Changing the HA Mode requires a reboot. After the details are set, click Reboot. Once the LoadMaster has rebooted, the HA Parameters menu option is available in the System Configuration section provided the role is not Non HA Mode. Configuring both units in the same HA Mode, for example, HA (First Mode) and HA (First) Mode, results in severe operational problems because; not only will both units be master, both units try to use the same IP address.

When logging in to the HA cluster, use the shared IP address to view and set the full functionality of the pair, apart from passwords and licensing. Logging in to the direct IP address of either one of the devices displays different menu options (see menus below). Logging into one of the LoadMasters directly is usually reserved for maintenance.

HA Parameters.png

 

H004.png

When a LoadMaster is in HA mode, the following screen appears when the HA Parameters menu option is selected:

126.png

After initial configuration, the HA parameters should not be modified unless both units in the HA pair are available and operating properly (if they are both showing green icons at the top of the WUI, with one LoadMaster in active mode and the other in standby).

HA Status

At the top of the screen, next to the time, icons denote the real-time status of the LoadMaster units in the cluster. There is an icon for each unit in the cluster. This status is maintained using an automatic ping between the units.

HA-green-green.png

Clicking these icons opens the management interface of the relevant HA partner.

The possible icons are: 

Green (with ‘A’)

HA Parameters_4.png

The unit is online and operational and the HA units are correctly paired.

The A in the middle of the square indicates that this is the master (active) unit.

Green (without ‘A’)

HA Parameters_5.png

The unit is online and operational and the HA units are correctly paired.

The absence of an ‘A’ in the middle of the square indicates that this is not the master unit (standby).

Red/Yellow

HA-red-sm

The partner unit is unreachable or turned off. It may be offline or misconfigured. The unit is not ready to take over. It may be offline or incorrectly paired.

Blue

HA-blue-sm

When the unit reboots more than three times in 5 minutes it enters a pacified state. In this state the machine is only accessible using the direct machine WUI (not the shared WUI) and it is not participating in any HA activity. Therefore, no changes from the master are received and it does not take over if the master fails. To remove the unit from the pacified state, fix the root cause of the health check failures, log in to the pacified LoadMaster through SSH or the console and reboot.

If a unit continuously reverts to a pacified state, check the network to see if CARP is being blocked.

Gray

HA-grey-sm

The machine is in an indeterminate state and may require a reboot to return to operation. A gray box often means the unit has not been set up in HA mode correctly. A gray box also appears for a few seconds during the initial HA configuration.

In some cases, it may mean both machines are active, that is, both are set to master, and something has gone seriously wrong.

Question marks 101.png The HA status is updating.
Both green (left box with 'A') 102.png Both units are up, unit 1 is master and unit 2 is standby.
Both green (right box with 'A') 103.png Both units are up, unit 1 is standby and unit 2 is master.
Left box green, right box red/yellow 104.png Unit 1 is up and currently master. Unit 1 cannot reach unit 2, or unit 2 is turned off.
Left box red/yellow, right box green 105.png Unit 2 is up and currently master. Unit 2 cannot reach unit 1, or unit 1 is turned off.
Left box gray, right box red/yellow 106.png HA setup is not complete on unit 1.
Left box red/yellow, right box gray 107.png HA setup is not complete on unit 2.
No HA icons

 

If the HA status squares are not appearing in the WUI, it probably means that HA is not enabled. Go to System Administration and select the HA option. Ensure the HA Mode is set to either First or Second.

In HA mode, each LoadMaster has its own IP address that is used only for diagnostic purposes directly on the unit. The HA pair have a shared IP address over which the WUI is used to configure and manage the pair as a single entity.

There are a number of prerequisites that must be in place for HA to function correctly. Refer to the Prerequisites section for a list of these prerequisites.

 HA Mode

If using a single LoadMaster, select NonHA Mode. When setting up HA mode, one LoadMaster must be set to HA (First) Mode and the other HA (Second) Mode. HA does not operate if both units have the same HA Mode.

 HA Timeout

 CARP requests are sent every second from the master. The value selected in the HA Timeout drop-down list is the time that the master machine must be unavailable before a switchover occurs. With this option, the time it takes a HA cluster to detect a failure can be adjusted from 3 seconds to 15 seconds in 3 second increments. The default value is 9 seconds. A lower value detects failures sooner, whereas a higher value prevents HA from failing over too soon if there is a delay when receiving CARP.

To set this option, follow the steps below:

1. Select System Configuration > HA Parameters.

2. Select the preferred value in the HA Timeout drop-down list.

 HA Initial Wait Time

The length of time after the initial boot of a LoadMaster, before the machine decides that it should become active. If the partner machine is running, this value is ignored. You can change this value to mitigate the time taken for some intelligent switches to detect that the LoadMaster has started and to bring up the link.

 HA Virtual ID

 When using multiple HA LoadMaster clusters (or other devices using CARP-like protocols) on the same network, this value uniquely identifies each cluster so that there are no potential unwanted interactions.

Kemp highly recommends using a higher value than 10 because any other HA pair using the same ID could interfere with HA operations.

As of the 7.2.36 release, the LoadMaster selects a virtual ID based on the shared IP address of the first configured interface (the last 8 bits). It is selected and displayed once both the shared address and the partner address are set. You can change the value to whatever you want (in the range 1 – 255) or you can keep it at the value it already selected. Ensure the virtual ID is unique on each LoadMaster on the network.

You can find the HA Virtual ID in the LoadMaster WUI by going to System Configuration > HA Parameters.

Switch to Preferred Server

 By default, neither partner in a HA cluster has priority. When a machine restarts after a failover, the machine becomes the standby and stays in that state until it is forced to master. Specifying a preferred host means that when this machine restarts, it always tries to become master and the partner reverts to standby mode.

When set to Prefer First HA, if the LoadMaster fails over, the master reverts to HA1 when HA1 comes back online.

When set to Prefer Second HA, if the LoadMaster fails over, the master reverts to HA2 when HA2 comes back online.

When No Preferred Host is selected, if there is a failover on the LoadMaster, the unit that becomes master remains as master (failback does not happen).

To change this option, follow the steps below in the LoadMaster WUI:

1. In the main menu, select Local Administration > HA Parameters.

2. Select the relevant option from the Switch to Preferred Server drop-down list.

Some connections may be dropped during the switchover if a preferred host is specified.

For normal operating conditions, Kemp recommends selecting No Preferred Host.

 HA Update Interface

The interface used to synchronize the entire HA configuration within the HA cluster. Synchronization occurs every two minutes. The information is synchronized over SSH port 6973.

Force Partner Update

Immediately forces the configuration from the active to standby unit without waiting for a normal update. This option is only available if both units can see each other in an active/standby scenario.

 Inter HA L4 TCP Connection Updates

 When using L4 services, if updates are enabled it allows L4 persistence to be maintained across a HA failover. This option is ignored for L7 services.

 Inter HA L7 Persistency Updates

 When using L7 services, if this option is enabled it allows persistence information to be shared between the HA partners. If a HA failover occurs, the persistence information is not lost.

Enabling this option can have a significant performance impact.

HA Multicast Interface

The network interface used for multicast traffic, which is used to synchronize Layer 4 and Layer 7 traffic when Inter HA Updates are enabled.

You can select the interface to send and receive inter-HA traffic from within the WUI of the shared IP address:

1. In the main menu, select System Configuration > HA Parameters.

2. The HA Update Interface setting is used for sending HA configuration updates using TCP/6973 between units. Modify it if needed.

If you have enabled L7 persistency updates or L4 TCP connection updates, an additional HA Multicast Interface option also becomes available.

5 Troubleshooting

This section outlines troubleshooting steps for some common HA-related problems. If further help is needed, please contact Kemp Support.

5.1 General Troubleshooting Tips

General HA troubleshooting steps are below:

  • Check that the IP settings for the Interface, Partner and Shared IP address are correct. These settings can be found in System Configuration > Interfaces.
  • Log in to each of the single HA interface addresses and ensure the HA parameters are correct (Local Administration > HA Parameters):
    • Ensure that the HA pair has one unit in HA (First) Mode and another unit in HA (Second) Mode.
    • Ensure that both units are on the same protocol and HA ID.
  • Ensure that all of the IP addresses are available and are not in use by another device. IP conflict causes numerous problems.
  • Shut down one or both LoadMasters and try to ping the IP address of each unit. If there is an answer, another device is using that IP address. Try the ‘arp -a’ or ‘netstat’ commands to find out more information on what device that is.
  • Set the HA Virtual ID (in Local Administration > HA Parameters) to something other than 1. The further up the range the better – avoid numbers from 1 to 10 because other HA pairs may have those IDs and use Virtual Router Redundancy Protocol (VRRP).

The Virtual ID can conflict with any device on the network which is using VRRP. If there are multiple HA clusters on the same network, they must also have different Virtual IDs.

  • Check that the time of both units are in sync and if they are not, ensure that Network Time Protocol (NTP) is configured and running on both units.
  • Ensure there are no Virtual Services using TCP and port 6973 on the interface where synchronization is configured.
  • Ensure there are no Virtual Services on either of the HA individual addresses.
  • Ensure there are no Virtual Services using TCP and port 22 on a LoadMaster interface port.

5.2 Confirm Settings

If you are experiencing problems with HA, confirm that:

  • The two LoadMasters are on the same subnet.
    127.png
  • For each Network Interface Card (NIC) that has Use for HA checks enabled, the link status is showing as connected.
  • The two units can ping each other and their default gateway. (The ping options are available in System Configuration > Logging Options > System Log Files > Debug Options.)
    170.png
  • Both units have the same time. Set both units to use the same NTP server and correct time zone. (The date and time options are available in System Configuration > System Administration > Date/Time).
    128.png
  • Unit 1 is set to HA (First) Mode.
    Unit 2 is set to HA (Second) Mode.
    The HA Virtual ID is the same on both units.

 References

While the instructions above provide a basic overview of how to deploy a LoadMaster HA Pair in CenturyLink Cloud, it is not designed to be a comprehensive guide. This section identifies some of many guides published on the resources section of our website. Unless otherwise specified, the following documents can be found at http://kemptechnologies.com/loadmaster-documentation.

Licensing, Feature Description

Web User Interface (WUI), Configuration Guide

Last Updated Date

This document was last updated on 29 January 2019.

Was this article helpful?

0 out of 0 found this helpful

Comments