ESP Logs

Updated: September 14, 2021 15:05

1 Introduction

This Technical Note provides supplementary information about the Edge Security Pack (ESP) logs in the Kemp LoadMaster. For further information on ESP in general, refer to the ESP Feature Description on the Kemp Documentation Page.

1.1 Related Firmware Version

Published with LMOS version 7.2.54 LTS. This document has not required substantial changes since 7.2.54 LTS. However, the content is in sync with the latest LoadMaster LTS firmware.

2 ESP SSO Debug Logs

ESP SSO debug logs are extensive. The primary purpose of these logs is to provide deep insight into processing and developer-level debugging information. While these logs are not documented, they are verbose in nature. They can be examined for information and parsed where necessary.

These logs are debug level and are disabled by default under normal operating conditions.

Generally, these logs are only enabled in collaboration with Kemp Customer Support personnel, to provide assistance with troubleshooting problematic flows.

3 ESP Extended Logs

These logs are generated from the L7 layer of the LoadMaster system. They provide insight into ESP and security-related events on the system. The format of these logs rarely change, unless there is a specific request to add extra information (which typically would be new data at the end of the string).

Three identifiers are used:

L7_LOG_CONN
L7_LOG_USER
L7_LOG_SECURITY

These map to the corresponding files on the system:

/var/log/userlog/connection
/var/log/userlog/user
/var/log/userlog/security

For more information on each of the log types, refer to the sections below.

3.1 Connection Logs

The connection logs provide information relating to the client, Virtual Service, Real Server, and the nature of the connection (if SSL is in use or not).

Format:

SSL accept on "VSIP:Port" from "Client IP:Port"

Format:

Connect from "ClientIP:Port" to "RSIP:Port" using "VSIP:Port"

3.2 User Logs

User logs reflect the activity of the user. The logs have the following format.

Format:

"VSIP:Port" ("RSIP:Port") User "USERNAME" requested|attempted "HTTP METHOD" "URI" "USERAGENT"

Where:

USERNAME reflects the user

The log indicates what the user requested OR attempted

HTTP METHOD reflects the HTTP method used, for example, GET or POST

URI comprises of http or https, the host being accessed, and the path and query as presented

USERAGENT is the User Agent header from the HTTP request (if enabled to be included). To enable this, go to System Configuration > Miscellaneous Options > L7 Configuration in the LoadMaster Web User Interface (WUI) and tick the Include User Agent Header in User Logs check box.

The user logs also explicitly shows log off activity.

Format:

"VSIP:Port": User "USERNAME" logged off

For common activity events (for example, log on and access denied), or if a dialogue is required between the client and LoadMaster (for example, for two-factor authentication), the user logs capture this detail in a simple user log message.

Format:

"VSIP:Port": User "USERNAME" "MESSAGE" from "HOST"

Where the MESSAGE can be:

logged on
denied access
blocked access
requires passphrase
requires re-enter passphrase
requires pin
requires re-enter pin
requires password reset

You can also generate user logs in Common Event Format (CEF).

To enable the CEF log format, go to System Configuration > Miscellaneous Options > L7 Configuration and select the Use CEF Log Format check box. CEF log format is easily consumable for Security Information and Event Management (SIEM) tools, such as; Splunk, SolarWinds, LogRhythm, AlienVault, and so on.

The following log headers appear in the user logs when the CEF format is enabled:

vs
event type
source ip
source port
user
user agent
request method
request url

For example:

================================================================================

files:

user-20200313.gz

================================================================================

<133>1 2020-03-12T09:08:26-0400 JWTest l7log - - [meta sequenceId="0"]

10.0.70.141:80: User ruth logged on from 10.0.11.113

<133>1 2020-03-12T09:08:26-0400 JWTest l7log - - [meta sequenceId="0"]

10.0.70.141:80: (10.0.11.113:52896) User 'ruth' requested GET

http://10.0.70.141/ (User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)

AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36

Edge/18.17763)

<133>1 2020-03-12T09:08:29-0400 JWTest l7log - - [meta sequenceId="0"]

10.0.70.141:80: (10.0.11.113:52897) User 'kempqaesp\ruth' requested GET

http://10.0.70.141/ (User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)

AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36

Edge/18.17763)

2020-03-12T11:17:00-04:00 JWTest l7log: 10.0.70.141:80: User ruth logged on

from 10.0.11.113

2020-03-12T11:17:00-04:00 JWTest l7log: 10.0.70.141:80: (10.0.11.113:57532)

User 'ruth' requested GET http://10.0.70.141/ (User-Agent: Mozilla/5.0

(Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)

Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763)

2020-03-12T11:29:55-04:00 JWTest l7log: CEF:0|Kemp|LM|1.0|9|Access

Denied|6|vs=10.0.70.141:80 event=Access Denied srcip=10.0.11.113 user=ruth

msg=denied access

2020-03-12T11:29:59-04:00 JWTest l7log: CEF:0|Kemp|LM|1.0|8|Logged

on|1|vs=10.0.70.141:80 event=Logged on srcip=10.0.11.113 user=ruth msg=logged

2020-03-12T11:29:59-04:00 JWTest l7log:

CEF:0|Kemp|LM|1.0|11|Request|1|vs=10.0.70.141:80 event=Request

srcip=10.0.11.113 srcport=57703 method=GET url=http://10.0.70.141/ user=ruth

2020-03-12T11:31:38-04:00 JWTest l7log:

CEF:0|Kemp|LM|1.0|11|Request|1|vs=10.0.70.133:443 event=Request

srcip=10.0.11.113 srcport=57782 method=GET url=https://mail.kempqaesp.net/

2020-03-12T11:31:38-04:00 JWTest l7log:

CEF:0|Kemp|LM|1.0|11|Request|1|vs=10.0.70.133:443 event=Request

srcip=10.0.11.113 srcport=57783 method=GET url=https://mail.kempqaesp.net/owa/

2020-03-12T11:31:38-04:00 JWTest l7log:

CEF:0|Kemp|LM|1.0|11|Request|1|vs=10.0.70.133:443 event=Request

srcip=10.0.11.113 srcport=57785 method=GET

url=https://mail.kempqaesp.net/owa/prem/15.1.544.27/scripts/boot.0.mouse.js

2020-03-12T11:31:38-04:00 JWTest l7log:

CEF:0|Kemp|LM|1.0|11|Request|1|vs=10.0.70.133:443 event=Request

srcip=10.0.11.113 srcport=57787 method=GET

url=https://mail.kempqaesp.net/owa/prem/15.1.544.27/scripts/boot.2.mouse.js

2020-03-12T11:31:38-04:00 JWTest l7log:

CEF:0|Kemp|LM|1.0|11|Request|1|vs=10.0.70.133:443 event=Request

srcip=10.0.11.113 srcport=57786 method=GET

url=https://mail.kempqaesp.net/owa/prem/15.1.544.27/scripts/boot.3.mouse.js

2020-03-12T11:31:38-04:00 JWTest l7log:

CEF:0|Kemp|LM|1.0|11|Request|1|vs=10.0.70.133:443 event=Request

srcip=10.0.11.113 srcport=57784 method=GET

url=https://mail.kempqaesp.net/owa/prem/15.1.544.27/scripts/boot.1.mouse.js

2020-03-12T11:31:38-04:00 JWTest l7log:

CEF:0|Kemp|LM|1.0|11|Request|1|vs=10.0.70.133:443 event=Request

srcip=10.0.11.113 srcport=57788 method=GET

url=https://mail.kempqaesp.net/owa/prem/15.1.544.27/resources/styles/fonts/of

fice365icons.woff

2020-03-12T11:34:35-04:00 JWTest l7log: CEF:0|Kemp|LM|1.0|9|Access

Denied|6|vs=10.0.70.134:443 event=Access Denied srcip=10.0.11.113

user=administrator msg=denied access

2020-03-12T11:34:40-04:00 JWTest l7log: CEF:0|Kemp|LM|1.0|8|Logged

on|1|vs=10.0.70.134:443 event=Logged on srcip=10.0.11.113 user=administrator

msg=logged on

2020-03-12T11:34:40-04:00 JWTest l7log: