Skip to Main Content

Java Security

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Kerberos RC4-HMAC Checksum Failed using Java 6 and Windows 2003/XP

843810Aug 14 2006 — edited Aug 23 2006
I am trying to get my IE6 client (running on XP) to authenticate to my JBoss server (on 2003) using the Active Directory on a 2003 box. I am using Java 6 Beta 2. My krb5.ini file is:
[libdefaults]
	default_realm = DEVEL.OPENROADSCONSULTING.COM
	default_tgs_enctypes = RC4-HMAC
	default_tkt_enctypes = RC4-HMAC

[kadmin]
        default_keys = v5 arcfour-hmac-md5

[realms]
	DEVEL.OPENROADSCONSULTING.COM = {
		kdc = interchange
		kdc = 192.168.100.101
		admin_server = interchange
		default_domain = devel.openroadsconsulting.com
	}

[domain_realm]
	.devel.openroadsconsulting.com = DEVEL.OPENROADSCONSULTING.COM

[appdefaults]
	autologin = true
	forward = true
	forwardable = true
	encrypt = true
My JBoss authenticates itself with the AD and my IE6 does the same. However, when I try to have the IE client authenticate with JBoss, I get the following error:
2006-08-14 11:48:44,920 INFO  [STDOUT] Entered Krb5Context.acceptSecContext with state=STATE_NEW
2006-08-14 11:48:44,920 INFO  [STDOUT] >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
2006-08-14 11:48:44,920 ERROR [STDERR] Checksum failed !
2006-08-14 11:48:44,920 ERROR [STDERR] jcifs.spnego.AuthenticationException: Error performing Kerberos authentication: java.lang.reflect.InvocationTargetException
2006-08-14 11:48:44,920 ERROR [STDERR] 	at jcifs.spnego.Authentication.processKerberos(Authentication.java:447)
2006-08-14 11:48:44,920 ERROR [STDERR] 	at jcifs.spnego.Authentication.processSpnego(Authentication.java:346)
2006-08-14 11:48:44,920 ERROR [STDERR] 	at jcifs.spnego.Authentication.process(Authentication.java:235)
2006-08-14 11:48:44,920 ERROR [STDERR] 	at org.jboss.web.tomcat.security.NegotiateUtil.extractUserId(NegotiateUtil.java:161)
2006-08-14 11:48:44,920 ERROR [STDERR] 	at org.jboss.web.tomcat.security.HttpServletRequestResponseValve.authenticate(HttpServletRequestResponseValve.java:98)
2006-08-14 11:48:44,920 ERROR [STDERR] 	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
2006-08-14 11:48:44,920 ERROR [STDERR] 	at org.jboss.web.tomcat.security.HttpServletRequestResponseValve.invoke(HttpServletRequestResponseValve.java:70)
2006-08-14 11:48:44,920 ERROR [STDERR] 	at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:74)
2006-08-14 11:48:44,920 ERROR [STDERR] 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
2006-08-14 11:48:44,920 ERROR [STDERR] 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at java.lang.Thread.run(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] Caused by: java.lang.reflect.InvocationTargetException
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at java.lang.reflect.Method.invoke(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at jcifs.spnego.Authentication.processKerberos(Authentication.java:430)
2006-08-14 11:48:44,998 ERROR [STDERR] 	... 16 more
2006-08-14 11:48:44,998 ERROR [STDERR] Caused by: java.security.PrivilegedActionException: java.lang.reflect.InvocationTargetException
2006-08-14 11:48:44,998 ERROR [STDERR] 	at java.security.AccessController.doPrivileged(Native Method)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at javax.security.auth.Subject.doAsPrivileged(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	... 21 more
2006-08-14 11:48:44,998 ERROR [STDERR] Caused by: java.lang.reflect.InvocationTargetException
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at java.lang.reflect.Method.invoke(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at jcifs.spnego.Authentication$ServerAction.run(Authentication.java:517)
2006-08-14 11:48:44,998 ERROR [STDERR] 	... 23 more
2006-08-14 11:48:44,998 ERROR [STDERR] Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	... 28 more
2006-08-14 11:48:44,998 ERROR [STDERR] Caused by: KrbException: Checksum failed
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.krb5.internal.crypto.ArcFourHmacEType.decrypt(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.krb5.internal.crypto.ArcFourHmacEType.decrypt(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.krb5.EncryptedData.decrypt(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.krb5.KrbApReq.authenticate(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.krb5.KrbApReq.<init>(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.jgss.krb5.InitSecContextToken.<init>(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	... 31 more
2006-08-14 11:48:44,998 ERROR [STDERR] Caused by: java.security.GeneralSecurityException: Checksum failed
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.krb5.internal.crypto.dk.ArcFourCrypto.decrypt(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	at sun.security.krb5.internal.crypto.ArcFourHmac.decrypt(Unknown Source)
2006-08-14 11:48:44,998 ERROR [STDERR] 	... 37 more
I loaded Ethereal and got the following authentication packet via HTTP:
No.     Time        Source                Destination           Protocol Info
   4310 4.782602    192.168.100.125       192.168.100.127       HTTP     GET /VicadsAdmin/GetFrontPage.event HTTP/1.1

Frame 4310 (686 bytes on wire, 686 bytes captured)
Ethernet II, Src: Dell_a6:00:f2 (00:13:72:a6:00:f2), Dst: Dell_63:73:e5 (00:13:72:63:73:e5)
Internet Protocol, Src: 192.168.100.125 (192.168.100.125), Dst: 192.168.100.127 (192.168.100.127)
Transmission Control Protocol, Src Port: 1081 (1081), Dst Port: 8080 (8080), Seq: 1461, Ack: 1, Len: 632
Reassembled TCP Segments (2092 bytes): #4309(1460), #4310(632)
Hypertext Transfer Protocol
    GET /VicadsAdmin/GetFrontPage.event HTTP/1.1\r\n
        Request Method: GET
        Request URI: /VicadsAdmin/GetFrontPage.event
        Request Version: HTTP/1.1
    Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, */*\r\n
    Accept-Language: en-us\r\n
    Accept-Encoding: gzip, deflate\r\n
    User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)\r\n
    Host: vicads0:8080\r\n
    Connection: Keep-Alive\r\n
    Authorization: Negotiate YIIFDgYGKwYBBQUCoIIFAjCCBP6gJDAiBgkqhkiC9xIBAgIGCSqGSIb3EgECAgYKKwYBBAGCNwICCqKCBNQEggTQYIIEzAYJKoZIhvcSAQICAQBuggS7MIIEt6ADAgEFoQMCAQ6iBwMFACAAAACjggPUYYID0DCCA8ygAwIBBaEfGx1ERVZFTC5PUEVOUk9BRFNDT05TVUxUSU5HLkNPTa
        GSS-API Generic Security Service Application Program Interface
            OID: 1.3.6.1.5.5.2 (SPNEGO - Simple Protected Negotiation)
            SPNEGO
                negTokenInit
                    mechTypes: 3 items
                        Item: 1.2.840.48018.1.2.2 (MS KRB5 - Microsoft Kerberos 5)
                        Item: 1.2.840.113554.1.2.2 (KRB5 - Kerberos 5)
                        Item: 1.3.6.1.4.1.311.2.2.10 (NTLMSSP - Microsoft NTLM Security Support Provider)
                    mechToken: 608204CC06092A864886F71201020201006E8204BB308204...
                    krb5_blob: 608204CC06092A864886F71201020201006E8204BB308204...
                        KRB5 OID: 1.2.840.113554.1.2.2 (KRB5 - Kerberos 5)
                        krb5_tok_id: KRB5_AP_REQ (0x0001)
                        Kerberos AP-REQ
                            Pvno: 5
                            MSG Type: AP-REQ (14)
                            Padding: 0
                            APOptions: 20000000 (Mutual required)
                                .0.. .... .... .... .... .... .... .... = Use Session Key: Do NOT use the session key to encrypt the ticket
                                ..1. .... .... .... .... .... .... .... = Mutual required: MUTUAL authentication is REQUIRED
                            Ticket
                                Tkt-vno: 5
                                Realm: DEVEL.OPENROADSCONSULTING.COM
                                Server Name (Service and Instance): HTTP/vicads0.devel.openroadsconsulting.com
                                    Name-type: Service and Instance (2)
                                    Name: HTTP
                                    Name: vicads0.devel.openroadsconsulting.com
                                enc-part rc4-hmac
                                    Encryption type: rc4-hmac (23)
                                    Kvno: 2
                                    enc-part: 05709AD578CD120E1292C1123131A078DEA84E68D6DE4AE8...
                            Authenticator rc4-hmac
                                Encryption type: rc4-hmac (23)
                                Authenticator data: 2AA7237E8F20DBA7090E3630FCF01EB7D29780CAEF6E8053...
    \r\n
I am suspicious of the APOptions not to use the session key but I cannot find how to change this (I have tried setting AllowTgtSessionKey to 1 but this does not change anything).

I've been beeting my head against the wall for a solid 3 days, can anyone please help me?

Thanks, David
Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Sep 20 2006
Added on Aug 14 2006
8 comments
7,306 views