beeline Kerberos： unsupported key type found the default TGT: 18

最新推荐文章于 2023-04-04 17:24:42 发布

tom_fans

最新推荐文章于 2023-04-04 17:24:42 发布

阅读量2.6k

点赞数

本文链接：https://blog.csdn.net/tom_fans/article/details/89197875

版权

新的环境Kerberos设置好之后，通过JAVA代码使用JDBC测试连接HIVE, IMPALA都不存在问题，但是使用beeline连接HIVE却报错。下面beeline中的principal=hive/tsczbdnndev1.trinasolar.com@TRINASOLAR.COM是指HIVE的主机的kerberos用户

export HADOOP_OPTS="-Dsun.security.krb5.debug=true"

Beeline version 1.1.0-cdh5.10.2 by Apache Hive
beeline> !connect jdbc:hive2://tsczbdnndev1.trinasolar.com:10000/default;principal=hive/tsczbdnndev1.trinasolar.com@TRINASOLAR.COM
scan complete in 1ms
Connecting to jdbc:hive2://tsczbdnndev1.trinasolar.com:10000/default;principal=hive/tsczbdnndev1.trinasolar.com@TRINASOLAR.COM
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
>>>KinitOptions cache name is /tmp/krb5cc_0
>>>DEBUG <CCacheInputStream>  client principal is hive@TRINASOLAR.COM
>>>DEBUG <CCacheInputStream> server principal is krbtgt/TRINASOLAR.COM@TRINASOLAR.COM
>>>DEBUG <CCacheInputStream> key type: 18
>>>DEBUG <CCacheInputStream> auth time: Wed Apr 10 21:07:25 CST 2019
>>>DEBUG <CCacheInputStream> start time: Wed Apr 10 21:07:25 CST 2019
>>>DEBUG <CCacheInputStream> end time: Thu Apr 11 21:07:25 CST 2019
>>>DEBUG <CCacheInputStream> renew_till time: Wed Apr 17 21:07:25 CST 2019
>>> CCacheInputStream: readFlags()  FORWARDABLE; RENEWABLE; INITIAL;
>>>DEBUG <CCacheInputStream>  client principal is hive@TRINASOLAR.COM
>>>DEBUG <CCacheInputStream> server principal is X-CACHECONF:/krb5_ccache_conf_data/fast_avail/krbtgt/TRINASOLAR.COM@TRINASOLAR.COM
>>>DEBUG <CCacheInputStream> key type: 0
>>>DEBUG <CCacheInputStream> auth time: Thu Jan 01 08:00:00 CST 1970
>>>DEBUG <CCacheInputStream> start time: null
>>>DEBUG <CCacheInputStream> end time: Thu Jan 01 08:00:00 CST 1970
>>>DEBUG <CCacheInputStream> renew_till time: null
>>> CCacheInputStream: readFlags() 
>>> unsupported key type found the default TGT: 18
19/04/10 22:46:03 [main]: ERROR transport.TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
        at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
        at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
        at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
        at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
        at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
        at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
        at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:203)
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:168)
        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:187)
        at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:146)
        at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:211)
        at org.apache.hive.beeline.Commands.connect(Commands.java:1499)
        at org.apache.hive.beeline.Commands.connect(Commands.java:1394)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52)
        at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1128)
        at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1167)
        at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1003)
        at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:915)
        at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:511)
        at org.apache.hive.beeline.BeeLine.main(BeeLine.java:494)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
        at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
        at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
        at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
        at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
        ... 35 more
Unknown HS2 problem when communicating with Thrift server.

上面错误有一段是unsupport type 18之类的，根据网友的意思，这个是AES256加密算法，但是256加密算法需要安装JCE POLICY，于是我去下载JCE的包，根据文档介绍拷贝2个包即，问题是仍然出错。

迫于无奈，我打算修改256为128，把KDC DATABASE删除重建，重新生成所有的key tab。

[root@tsczbddndev2 ~]# beeline
Beeline version 1.1.0-cdh5.10.2 by Apache Hive
beeline> !connect jdbc:hive2://tsczbdnndev1.trinasolar.com:10000/default;principal=hive/tsczbdnndev1.trinasolar.com@TRINASOLAR.COM
scan complete in 2ms
Connecting to jdbc:hive2://tsczbdnndev1.trinasolar.com:10000/default;principal=hive/tsczbdnndev1.trinasolar.com@TRINASOLAR.COM
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
>>>KinitOptions cache name is /tmp/krb5cc_0
>>>DEBUG <CCacheInputStream>  client principal is hive@TRINASOLAR.COM
>>>DEBUG <CCacheInputStream> server principal is krbtgt/TRINASOLAR.COM@TRINASOLAR.COM
>>>DEBUG <CCacheInputStream> key type: 17
>>>DEBUG <CCacheInputStream> auth time: Wed Apr 10 22:46:42 CST 2019
>>>DEBUG <CCacheInputStream> start time: Wed Apr 10 22:46:42 CST 2019
>>>DEBUG <CCacheInputStream> end time: Thu Apr 11 22:46:42 CST 2019
>>>DEBUG <CCacheInputStream> renew_till time: Wed Apr 17 22:46:42 CST 2019
>>> CCacheInputStream: readFlags()  FORWARDABLE; RENEWABLE; INITIAL;
>>>DEBUG <CCacheInputStream>  client principal is hive@TRINASOLAR.COM
>>>DEBUG <CCacheInputStream> server principal is X-CACHECONF:/krb5_ccache_conf_data/fast_avail/krbtgt/TRINASOLAR.COM@TRINASOLAR.COM
>>>DEBUG <CCacheInputStream> key type: 0
>>>DEBUG <CCacheInputStream> auth time: Thu Jan 01 08:00:00 CST 1970
>>>DEBUG <CCacheInputStream> start time: null
>>>DEBUG <CCacheInputStream> end time: Thu Jan 01 08:00:00 CST 1970
>>>DEBUG <CCacheInputStream> renew_till time: null
>>> CCacheInputStream: readFlags() 
Found ticket for hive@TRINASOLAR.COM to go to krbtgt/TRINASOLAR.COM@TRINASOLAR.COM expiring on Thu Apr 11 22:46:42 CST 2019
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for hive@TRINASOLAR.COM to go to krbtgt/TRINASOLAR.COM@TRINASOLAR.COM expiring on Thu Apr 11 22:46:42 CST 2019
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
default etypes for default_tgs_enctypes: 17.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType
>>> KdcAccessibility: reset
>>> KrbKdcReq send: kdc=tsczbddbprd4.trinasolar.com TCP:88, timeout=3000, number of retries =3, #bytes=665
>>> KDCCommunication: kdc=tsczbddbprd4.trinasolar.com TCP:88, timeout=3000,Attempt =1, #bytes=665
>>>DEBUG: TCPClient reading 652 bytes
>>> KrbKdcReq send: #bytes read=652
>>> KdcAccessibility: remove tsczbddbprd4.trinasolar.com
>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType
>>> KrbApReq: APOptions are 00100000 00000000 00000000 00000000
>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType
Krb5Context setting mySeqNumber to: 845672618
Created InitSecContextToken:
0000: 01 00 6E 82 02 35 30 82   02 31 A0 03 02 01 05 A1  ..n..50..1......
0010: 03 02 01 0E A2 07 03 05   00 20 00 00 00 A3 82 01  ......... ......
0020: 57 61 82 01 53 30 82 01   4F A0 03 02 01 05 A1 10  Wa..S0..O.......
0030: 1B 0E 54 52 49 4E 41 53   4F 4C 41 52 2E 43 4F 4D  ..TRINASOLAR.COM
0040: A2 2E 30 2C A0 03 02 01   00 A1 25 30 23 1B 04 68  ..0,......%0#..h
0050: 69 76 65 1B 1B 74 73 63   7A 62 64 6E 6E 64 65 76  ive..tsczbdnndev
0060: 31 2E 74 72 69 6E 61 73   6F 6C 61 72 2E 63 6F 6D  1.trinasolar.com
0070: A3 82 01 04 30 82 01 00   A0 03 02 01 11 A1 03 02  ....0...........
0080: 01 02 A2 81 F3 04 81 F0   A5 99 C3 90 35 61 2D 46  ............5a-F
0090: C2 FB 63 18 E1 FF 14 94   79 9D C8 56 C1 3C 28 7C  ..c.....y..V.<(.
00A0: 98 A2 E8 52 B7 90 65 1F   85 74 18 D5 7C 9B 25 94  ...R..e..t....%.
00B0: 07 FD 29 57 80 BE 81 2A   67 CE 50 2F 56 E0 AD 13  ..)W...*g.P/V...
00C0: 31 04 2A 11 DB 91 D1 F9   3A 3F A9 11 AE 8E F1 13  1.*.....:?......
00D0: E9 6C 3F 32 27 A9 A0 B8   97 5B 10 F4 01 5F 2B 50  .l?2'....[..._+P
00E0: 0A F5 A4 08 3D 4F 2F DD   E7 E4 58 F2 A1 02 34 70  ....=O/...X...4p
00F0: A3 22 E9 CA 58 8C 71 A3   71 0B 00 40 0A C4 33 B9  ."..X.q.q..@..3.
0100: 1A D2 D1 C2 01 07 26 3B   3F D2 BA C4 E8 AE 58 BA  ......&;?.....X.
0110: 81 5C B2 42 42 6E 66 09   84 C4 44 93 E1 3A 88 34  .\.BBnf...D..:.4
0120: FF 9F D4 52 3A D3 00 B5   D2 E9 5A F9 95 E9 38 51  ...R:.....Z...8Q
0130: 9E 5F A8 A8 DD A8 3F 97   97 33 21 E9 53 33 31 3D  ._....?..3!.S31=
0140: EA A8 A8 5A AB F8 A8 5B   83 3B A5 55 A9 68 A1 BD  ...Z...[.;.U.h..
0150: D2 EA A7 D8 08 48 6A DA   C9 38 70 CA B6 B8 97 51  .....Hj..8p....Q
0160: 04 36 1D E3 81 D7 AB 13   B8 CF 81 8E 91 FE C4 AE  .6..............
0170: BA A7 11 D0 43 8F 9A 92   A4 81 C0 30 81 BD A0 03  ....C......0....
0180: 02 01 11 A2 81 B5 04 81   B2 32 5C 2D 2C 35 B6 14  .........2\-,5..
0190: 6F 95 E5 76 2C 64 08 E9   B3 B4 1B D9 2F 36 C7 78  o..v,d....../6.x
01A0: 91 B1 A4 1E 78 4C 6C C9   FC B5 F6 AE 6F D1 6C C2  ....xLl.....o.l.
01B0: 37 38 65 E5 45 CF 3D E4   6F 65 0E 0C D3 3C 23 ED  78e.E.=.oe...<#.
01C0: CC DC 6A 6D 36 DD 89 47   E6 C0 31 C4 25 51 77 FE  ..jm6..G..1.%Qw.
01D0: 71 B0 F0 0A 45 FB 59 BA   28 08 22 A3 94 FC 7A 45  q...E.Y.(."...zE
01E0: C9 77 5C A8 C1 24 19 A9   41 21 85 BC BE 5D 61 59  .w\..$..A!...]aY
01F0: CF 75 2E 37 C1 07 0B EF   4B 9F 10 E3 1C F0 E0 85  .u.7....K.......
0200: 0E F5 EB 4B 2C 35 38 C9   A3 DB 1E 9B D7 EA 6D 14  ...K,58.......m.
0210: 28 61 41 7D DA 17 F4 01   2F 5C 9F 6B 9C AB 55 E8  (aA...../\.k..U.
0220: 77 33 8D 47 46 DC 90 C0   8B 76 2D 82 E3 5F 4D 9D  w3.GF....v-.._M.
0230: EC 10 E2 4E B1 26 F8 87   E2 36 DF                 ...N.&...6.

Entered Krb5Context.initSecContext with state=STATE_IN_PROCESS
>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType
Krb5Context setting peerSeqNumber to: 398446423
Krb5Context.unwrap: token=[05 04 01 ff 00 0c 00 00 00 00 00 00 17 bf cf 57 01 01 00 00 42 bb 38 4a fd e0 d1 ea b8 ab 38 34 ]
Krb5Context.unwrap: data=[01 01 00 00 ]
Krb5Context.wrap: data=[01 01 00 00 ]
Krb5Context.wrap: token=[05 04 00 ff 00 0c 00 00 00 00 00 00 32 67 f0 aa 01 01 00 00 0e 27 2a 91 83 3a 81 5b ae 3d 9e 5e ]
Connected to: Apache Hive (version 1.1.0-cdh5.10.2)
Driver: Hive JDBC (version 1.1.0-cdh5.10.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ

上面已经连接成功，从日志显示128加密type为17.