sqoop 报错之java.lang.InternalError

java.lang.InternalError

        at sun.security.ec.SunEC.initialize(Native Method)

        at sun.security.ec.SunEC.access$000(SunEC.java:49)

        at sun.security.ec.SunEC$1.run(SunEC.java:61)

        at sun.security.ec.SunEC$1.run(SunEC.java:58)

        at java.security.AccessController.doPrivileged(Native Method)

        at sun.security.ec.SunEC.<clinit>(SunEC.java:58)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

        at java.lang.Class.newInstance(Class.java:383)

        at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:221)

错误由于已经消除了 所以堆栈 可能不太一样

公司新弄的hadoop集群 安装好sqoop 之后执行sqoop脚本 发现以上问题

看这个错误决定很无奈 上hadoop源码中JobSubmitter.java这个类去查找了submitJobInternal这个方法

  // generate a secret to authenticate shuffle transfers
      if (TokenCache.getShuffleSecretKey(job.getCredentials()) == null) {
        KeyGenerator keyGen;
        try {
          keyGen = KeyGenerator.getInstance(SHUFFLE_KEYGEN_ALGORITHM);
          keyGen.init(SHUFFLE_KEY_LENGTH);
        } catch (NoSuchAlgorithmException e) {
          throw new IOException("Error generating shuffle secret key", e);
        }
        SecretKey shuffleKey = keyGen.generateKey();
        TokenCache.setShuffleSecretKey(shuffleKey.getEncoded(),
            job.getCredentials());
      }
      if (CryptoUtils.isEncryptedSpillEnabled(conf)) {
        conf.setInt(MRJobConfig.MR_AM_MAX_ATTEMPTS, 1);
        LOG.warn("Max job attempts set to 1 since encrypted intermediate" +
                "data spill is enabled");
      }

这段代码在异常堆栈中出现了 然后继续google

https://bugzilla.redhat.com/show_bug.cgi?id=1332867

在上面这个网页中 写了一个Test.java的测试类

import java.security.Security;

public class Test {
  public static void main(String args[]) throws SecurityException {
    Security.removeProvider("SunPKCS11-Solaris");
  }
}

然后javac Test.java

java Test

Exception in thread "main" java.lang.InternalError
	at sun.security.ec.SunEC.initialize(Native Method)
	at sun.security.ec.SunEC.access$000(SunEC.java:49)
	at sun.security.ec.SunEC$1.run(SunEC.java:61)
	at sun.security.ec.SunEC$1.run(SunEC.java:58)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.security.ec.SunEC.<clinit>(SunEC.java:58)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:221)
	at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206)
	at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187)
	at sun.security.jca.ProviderList.loadAll(ProviderList.java:282)
	at sun.security.jca.ProviderList.removeInvalid(ProviderList.java:299)
	at sun.security.jca.Providers.getFullProviderList(Providers.java:173)
	at java.security.Security.removeProvider(Security.java:440)
	at Test.main(Test.java:5)

然后发现错误异常堆栈基本上一致 所以确定应该就是这个文章里所述的东西

最后把原来的open-jdk换成了oracle的jdk

重新执行java Test 通过

然后找sqoop的java_home

发现sqoop是通过hadoop来运行的 所以直接修改hadoop的java_home即可

修改之后 通过了