ESP Kerberos SSO Debug Trace

 

Scope

This article demonstrates how you can troubleshoot Kerberos by analyzing the data in both the SSOMGR Debug traces and Wireshark. 

To collect the data, enable verbose Edge Security Pack (ESP) logging by navigating to:

Logging Options > System Log Files > Debug Options > SSO MGR Logging

To view the log data navigate to:

Logging Options > System Log Files > System Message File. 

For firmware older than 7.2.41, navigate to:

Logging Options > Extended Log Files  > SSO MGR Audit Logs >  SSO MGR > View.

 

 

1. Reverse DNS Lookup

Once a client has successfully completed front-fnd authentication, for example, Forms Based, NTLM, Client Certificates, or SAML, the LoadMaster requests a Kerberos ticket for the back-end server. 

Depending on what destination server is chosen from the Virtual Service pool, the LoadMaster must perform a reverse Domain Name System (DNS) lookup on the server. Depending on the hostname returned from the DNS, this name is used to form the Service Principal Name (SPN)

In the example below, the IP address is 10.110.30.5 and the hostname is sharepoint1.kemptest.com. Therefore, the SPN becomes: http/sharepoint1.kemptest.com@KEMPTEST.COM.

 

Feb 14 09:36:21 lb100 ssomgr: #32301# baseUserName: basename=|admin|
Feb 14 09:36:21 lb100 ssomgr: #32301# >>> kcd_get_user_ticket
Feb 14 09:36:21 lb100 ssomgr: #32301# >>>resolve_destination_address: Attempt to resolve destination [10.110.30.5][2]
Feb 14 09:36:21 lb100 ssomgr: #32301# <<<resolve_destination_address: Resolved destination host name [sharepoint1.kemptest.com]
Feb 14 09:36:21 lb100 ssomgr: #32301# kcd_get_user_ticket: user=[admin@kemptest.com] [basename=[admin]
Feb 14 09:36:21 lb100 ssomgr: #32301# kcd_get_user_ticket: Destination name=[http/sharepoint1.kemptest@KEMPTEST.com]
Feb 14 09:36:21 lb100 ssomgr: #32301# kcd_get_user_ticket: kcd_ticket:0x7fde4f1f2ef0 [65536/65536]
Feb 14 09:36:21 lb100 ssomgr: #32301# >>> get_impersonator_cred_handle
Feb 14 09:36:21 lb100 ssomgr: #32301# >>> get_impersonator_cred_handle - handle=0x7fde300008c0

 

Note: If you see Resolved destination host name followed by [0], it means that the LoadMaster was unable to perform a Reverse DNS Lookup. In this case, check your Reverse DNS Lookup records. 

You can also test by adding host file entries to the LoadMaster by navigating to:

System configuration > Host & DNS Configuration.

 

 

2. Contact Kerberos Server

In the example below, the LoadMaster attempts to retrieve a ticket on behalf of the user Admin, using Delegated name (kempkcd) configured in the server side configuration. 


Feb 14 09:36:21 lb100 ssomgr: #32301# kcd_get_user_ticket: Get a ticket on behalf of user admin
Feb 14 09:36:21 lb100 ssomgr: #32290# kcd_get_user_ticket: Credentials aquired
Feb 14 09:36:21 lb100 ssomgr: #32290# init_accept_sec_context(): Target name: [kempkcd@kemptest.com]
Feb 14 09:36:21 lb100 ssomgr: #32290# Target mech: [{ 1 3 6 1 5 5 2 }]
Feb 14 09:36:21 lb100 ssomgr: #32290# init_accept_sec_context(): Source name: [admin@KEMPTEST.COM]
Feb 14 09:36:21 lb100 ssomgr: #32290# init_accept_sec_context(): Source mech: [{ 1 2 840 113554 1 2 2 }]
Feb 14 09:36:21 lb100 ssomgr: #32290# Attribute urn:mspac: Authenticated Complete
Feb 14 09:36:21 lb100 ssomgr: #32290# Attribute urn:mspac:logon-info Authenticated Complete
Feb 14 09:36:21 lb100 ssomgr: #32290# Attribute urn:mspac:client-info Authenticated Complete
Feb 14 09:36:21 lb100 ssomgr: #32290# Attribute urn:mspac:upn-dns-info Authenticated Complete
Feb 14 09:36:21 lb100 ssomgr: #32290# Attribute urn:mspac:server-checksum Authenticated Complete
Feb 14 09:36:21 lb100 ssomgr: #32290# Attribute urn:mspac:privsvr-checksum Authenticated Complete
Feb 14 09:36:21 lb100 ssomgr: #32290# Proxy name: [kempkcd@KEMPTEST.COM]
Feb 14 09:36:21 lb100 ssomgr: #32290# Target name: [http/sharepoint.kemptest.com@KEMPTEST.COM]
Feb 14 09:36:21 lb100 ssomgr: #32290# Delegated name: [admin@kemptest.COM]
Feb 14 09:36:21 lb100 ssomgr: #32290# Delegated mech: [{ 1 2 840 113554 1 2 2 }]
Feb 14 09:36:21 lb100 ssomgr: #32290# encode_ticket: sz=2060 ticket=0x7fde4f255ef0 len=65536 buf_sz=65536

 

3. Successfully Retrieved Kerberos Ticket

If successful, you should see the Negotiate authorization header with a long random string of characters. 


Feb 14 09:36:21 lb100 ssomgr: #32290# <<< kcd_get_user_ticket - ret=0
Feb 14 09:36:21 lb100 ssomgr: #32290# kcd_reauth_thread: status and cookie are both OK
Feb 14 09:36:21 lb100 ssomgr: #32290# kcd_reauth_thread: up->kcd_data[Negotiate YIIF+wYJKoZIhvcSAQICAQBuggXqMIIF5qADAgEFoQMCAQ6iBwMFAAAAAACjggT+YYIE+jCCBPagAwIBBaEMGwpTVFJFRVQuQ09NoiowKKADAgEBoSEwHxsEaHR0cBsXZXhjaGFuZ2UyMDEzLnN0cmVldC5jb22jggSzMIIEr6ADAgESoQMCAQSiggShBIIEnag04R/bTzdmz+ZEjyNgYx9hm9gESS2aY3OwbxvT85NrqguM31y9HHBLRDehk/mfX13S0rEwtwyHT9EWwD5g7zNVpoeczaHN+waU74GjS7Th+WYrLHCNbaQK5v269n/L+s1E1M53QCARCSsoTYvfZDZDbLQNFNFDVBWR2JDE0s0iljupa+KBWampg//BPV0xLiUzQdKGYtScpec4buOS2eJmiaI7fcoj+OlnMnL47m2L/5IxjNC8QM5U1GdcLuSUSPdsnIdrinKjHsfXc8msyiXyWIbe2SnVxs18tPVVul0KGnzxAdB8X17RbKb7ViDVs3IZckF7azNqhwg3C0tBJlAys5ySQ3rXSLHBeF46Wyn8bLiGkLyVMQ6bRmb8fLPnEshakPrtuR+YuaZZ/uMEaeP7hIBQ2JkppX4xquAszoTwXOREgTjqL6BOdHGM9UKvevRQxa9x9Rod7kW532kUTc+ayB5S8DhW2FT79zvRhKmKXk8n2dsGLl2csiBC+bl9uMdCykFrWz+nl2DzHS8L+6tM0JjqxetmqNdQvuCMSLaBS1Upg2ZhR/2vNh641h7NzZSBNPXcfzionp1nRkk2jPYSmq45d+/c6/Jabied5PwvUYXur91ijjJ8+cnTmWZtQjN6I4EDW3Td/g9UnG/PzEzWivQN03lAjQwRa20K+HM4w4kOADw7DcEDErSF2RZsKjHh2cTvf4/sZlC1TY38l5BNHBLWu7bwnvChkjJAkxaC/A4
Feb 14 09:36:21 lb100 ssomgr: #32290# >>> release_sso_conf(0x6b52b0), calling function(kcd_reauth_thread)

 

If you receive the error message shown in the example below, ensure that the delegated account is configured correctly. Also ensure that you add the server name under the Delegated tab and not, for example, the name of your SharePoint site. 

Feb 14 09:36 lb100 ssomgr: #57508# kinit_domain: Getting credentials for kempkcd@kemptest.com
Feb 14 09:36 lb100 ssomgr: kinit_domain: Could not initialize credentials: error -1765328360

 

Feb 14 09:36 lb100 ssomgr: #24663# gss_init_sec_context: Unspecified GSS failure.  Minor code may provide more information

Feb 14 09:36 lb100 ssomgr: #24663# gss_init_sec_context: KDC can't fulfill requested option

 

 

4. Wireshark

If you think that your configuration is correct, perform a Transmission Control Protocal (TCP) trace and filter on your Kerberos server IP address.

To perform a TCP trace, navigate to:

Logging Options --> System Log files --> Debug Options --> TCP Dump. Select Interface = All and IP Address = <Domain Controller IP>.

 

When you open the Wireshark PCAP file, filter using the word Kerberos. 

Pay attention to the following information:

AS-REQ: This is where the client is authenticated and a ticket granting ticket (TGT) is retrieved. In this case your client is the KEMP Delegated User. 

TGS-REQ: Here, you present your TGT to Kerberos where it is verified against its database.

 KCD Error Codes and possible causes

For more information on Kerberos Constrained Delegation (KCD) error codes and possible causes, visit: Kerberos Error Codes

If you receive no errors in Wireshark, then try flushing the SSO Cache. 

Logging Options > System Log Files > Debug Options > Flush SSO Cache.  If it still doesn't work try rebooting the Load Master. 

Altering Kerberos SSO Settings.

If a change of credentials is required, you will need to flush SSO Cache. 

 

Related KBs

ESP Client Certificate SSO Debug Trace

ESP SAML SSO Debug Trace

Was this article helpful?

0 out of 0 found this helpful

Comments