Troubleshooting Real Server/Virtual Service unavailable
Introduction
One of the most important aspects of maintaining Application availability is ensuring the health of Real Servers for which Virtual Service Traffic is configured to be sent. The LoadMaster does this by utilising the configured Health Check and if this fails considering the Real Server unhealthy and taking it out service.
While the redundancy provided by A Load Balancer should ensure application availability, any unavailable real server should be investigated.
Like all Network Application issues there are a number of potential issues causing a Real Server to be down. Below are some suggested questions to as/ things to check to diagnose these issues:
1. Health of Real Server
The first thing to check is if the Real Server is actually healthy.Typically this involves examining the Real Server directly.
- Is the device up?
- Check Reachability of Server, Status of Virtual Machine etc
- Is the Application running?
- Can the application be accessed locally on the Server?
- Can I access the Real Server directly form my own client?
- Can I access the application by pointing my Client Directly at the Real Server (This may required local DNS update)
2. Check for appropriate Health Check
If all this looks good the next thing to examine is if the Health Check in use is appropriate.
- Is the LoadMaster Virtual Service using the correct Health Check?
- this can be found under Virtual Services>usedCan I successfully perform the same health check from my own PC/Client?
- At what level is the Health check failing ?
- Utilising the LoadMaster TCPDUMP feature this traffic can be reviewed.
3. Check LoadMaster to Real Server Traffic is allowed
- Is there a firewall blocking traffic?
- Check what firewall rules may be in place between the loadMaster and Real Server
- Is there a local firewall blocking traffic?
- Check the Local Device Firewall Rules.
- Is traffic allowed from both units (if using a HA pair of LoadMasters?)
- Utilising the LoadMaster TCPDUMP feature this traffic can be reviewed.
4. Other
- Duplicate IP Address. Is it possible there is another device using the same IP as my Real Server?
- In order to investigate check the arp table entries for the Real Server.
- Excessive Traffic on Real Server. Is it possible that the Real Server is handling excessive traffic loads and unable to process the Health check request?
- On the Application check the volume of connections being processed.