How to enable L25 debug traces from the LoadMaster's Web UI

How to enable L25 debug traces from the LoadMaster's Web UI using the Extended Logs.

Product: LoadMaster

Version: Any

Platform: Any

Application: HTTP/HTTPS

How to enable L25 debug traces to inspect HTTPS traffic in clear text when a packet capture is not possible through the LoadMaster

LM-1750

NOTE: Enabling L25 debug logs may expose sensitive information.

Extended Debug logs need to be enabled prior enabling L25 logs. This can be enabled by going to System Configuration > Logging Options > System Log Files > Debug Options > Enable Extended Debug.

L25 debug traces can be enabled by going to Virtual Services > View/Modify Services > Modify > Extended Debug > Full Debug + HTTP Headers.

By default, the logs will look like below:

2022-10-11T18:03:38+00:00 lb100 kernel: L7: ffff88806f389d10: RS 10.67.48.140:80 aconns 0 refcnt 2 weight 1000 2
2022-10-11T18:03:38+00:00 lb100 kernel: L7: ffff88806f389d10: Connecting from 10.67.48.160:7917 to 10.67.48.140:80 3
2022-10-11T18:03:38+00:00 lb100 kernel: L7: ffff88806f389d10: Connected 4
2022-10-11T18:03:38+00:00 lb100 kernel: L7: ffff88806f389d10: Conn: Request 1 ms Response 0 ms 5
2022-10-11T18:03:38+00:00 lb100 kernel: L7: ffff88806f389d10: Conn: dest RTT 315/266 us min 224 us 6
2022-10-11T18:03:38+00:00 lb100 kernel: L7: ffff88806f389d10: conn release 7

As shown above, even though L25 logs are enabled, the logs do not show the actual HTTP requests coming from the client. This is because L25 will only work if the LM is doing something to process the HTTP request. If we are operating at Kernel-level request processing then nothing is seen in the logs. Without invoking the L7 Engine, the LM will act as a simple request forwarder.

To get the L7 engine involved, Super HTTP or Active Cookie can be used as persistence methods, or even as simple as setting a HTTP error code in the Not Available Redirection Handling section (most options in Advanced Properties will invocke the L7 engine).

Once these changes are reflected, the logs will look like the following:

2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: SSL accept on 10.67.48.150:443 from 10.248.3.29:61806 (0) 2
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: Parse_http_header 3
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: User-Agent: curl/7.58.0 4
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: Super pkey 'curl/7.58.0' 5
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: Parse_http_header: finished 6
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: request: HEAD / 7
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: ### SOH: entries:3 length:77 8
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: ## Host: 10.67.48.150 9
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: ## User-Agent: curl/7.58.0 10
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: ## Accept: */* 11
2022-10-11T18:06:42+00:00 lb100 kernel: L7: ffff88807041cd10: ### EOH

https://support.kemptechnologies.com/hc/en-us/articles/6263906628109-Extended-L7-Debug