启动测试apache impala
加载环境配置
-
加载环境变量
由于编译impala的时候下载了很多相关依赖配置，但是有些可能需要使用我们自己的环境依赖需要进行部分修改
source /app/impala/bin/impala-config.sh -
修改自定义环境变量
由于impala依赖的项目过多，所以环境变量配置的不对，完全起不来，所以这么给出需要配置的环境变量，有自己搭建的自行修改,这里面最麻烦的就是CLASSPATH，我后面给个脚本输出一下这个CLASSPATH,这步很重要，因为假如没有这步，你会一直报java core dump
source /app/impala/bin/impala-config.sh获取所有依赖
for f in /app/impala/fe/target/dependency/*.jar; do export CLASSPATH=$CLASSPATH:$f done echo $CLASSPATH修改环境变量文件，将以下内容拷贝到一个sh文件，然后执行再启动impala
export IMPALA_HOME=/app/impala export HADOOP_HOME=/app/hadoop-3.1.2 export HADOOP_CONF_DIR=/etc/impala/conf.dist export HADOOP_INCLUDE_DIR=/app/hadoop-3.1.2/include export HADOOP_LIB_DIR=/app/hadoop-3.1.2/lib export MINI_DFS_BASE_DATA_DIR=/app/impala/hdfs-data export HIVE_HOME=/app/hive export HIVE_CONF_DIR=/etc/impala/conf.dist/ export HIVE_SRC_DIR=/app/impala/toolchain/cdh_components-1814051/hive-2.1.1-cdh6.x-SNAPSHOT/src export HBASE_HOME=/app/impala/toolchain/cdh_components-1814051/hbase-2.1.0-cdh6.x-SNAPSHOT/ export HBASE_CONF_DIR=/app/impala/fe/src/test/resources export SENTRY_HOME=/app/impala/toolchain/cdh_components-1814051/sentry-2.1.0-cdh6.x-SNAPSHOT export SENTRY_CONF_DIR=/app/impala/fe/src/test/resources export RANGER_HOME=/app/impala/toolchain/cdp_components-1617729/ranger-2.0.0.7.0.2.0-212-admin export RANGER_CONF_DIR=/app/impala/fe/src/test/resources export THRIFT_HOME=/app/impala/toolchain/thrift-0.9.3-p7 export HADOOP_LZO=/app/impala/../hadoop-lzo export IMPALA_LZO=/app/impala/../Impala-lzo export LIBHDFS_OPTS="-Djava.library.path=/app/impala/toolchain/cdh_components-1814051/hadoop-3.0.0-cdh6.x-SNAPSHOT//lib/native/ -XX:ErrorFile=/app/impala/logs/hs_err_pid%p.log -XX:MaxPermSize=128m -Djava.library.path=/app/impala/toolchain/cdh_components-1814051/hadoop-3.0.0-cdh6.x-SNAPSHOT//lib/native/ -XX:ErrorFile=/app/impala/logs/hs_err_pid%p.log -XX:MaxPermSize=128m" export JAVA_HOME=/usr/java/jdk1.8.0_201-amd64 export LD_LIBRARY_PATH=:/usr/java/jdk1.8.0_201-amd64/jre/lib/amd64:/usr/java/jdk1.8.0_201-amd64/jre/lib/amd64/server:/usr/java/jdk1.8.0_201-amd64/jre/lib/amd64/server:/app/impala/toolchain/cdh_components-1814051/hadoop-3.0.0-cdh6.x-SNAPSHOT//lib/native:/app/impala/toolchain/snappy-1.1.4/lib:/app/impala/../Impala-lzo/build:/app/impala/toolchain/gcc-4.9.2/lib64:/usr/java/jdk1.8.0_201-amd64/jre/lib/amd64:/usr/java/jdk1.8.0_201-amd64/jre/lib/amd64/server:/usr/java/jdk1.8.0_201-amd64/jre/lib/amd64/server:/app/impala/toolchain/cdh_components-1814051/hadoop-3.0.0-cdh6.x-SNAPSHOT//lib/native:/app/impala/toolchain/snappy-1.1.4/lib:/app/impala/../Impala-lzo/build:/app/impala/toolchain/gcc-4.9.2/lib64 export LD_PRELOAD=:/usr/java/jdk1.8.0_201-amd64/jre/lib/amd64/server/libjsig.so:/usr/java/jdk1.8.0_201-amd64/jre/lib/amd64/server/libjsig.so export MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar export IMPALA_CATALOG_SERVICE_HOST=sit1 export IMPALA_STATE_STORE_HOST=sit1 export IMPALA_STATE_STORE_PORT=24000 export IMPALA_BACKEND_PORT=22000 export IMPALA_LOG_DIR=/app/impala/logs/impala export IMPALA_CATALOG_ARGS=" --log_dir=${IMPALA_LOG_DIR} " export IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}" export IMPALA_SERVER_ARGS=" \ -log_dir=${IMPALA_LOG_DIR} \ -state_store_host=${IMPALA_STATE_STORE_HOST} \ -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \ -state_store_port=${IMPALA_STATE_STORE_PORT} \ -be_port=${IMPALA_BACKEND_PORT}" export ENABLE_CORE_DUMPS=false export JAVA_HOME=/usr/java/jdk1.8.0_201-amd64 export CLASSPATH=./:/app/impala/fe/target/impala-frontend-0.1-SNAPSHOT.jar:/app/hive//lib:/usr/java/jdk1.8.0_201-amd64/lib:/usr/java/jdk1.8.0_201-amd64/jre/lib:/app/spark//lib:/app/hadoop-3.1.2/share/hadoop/common:/app/hadoop-3.1.2/share/hadoop/yarn:/app/spark//lib:/app/impala/fe/target/dependency/accessors-smart-1.2.jar:/app/impala/fe/target/dependency/activation-1.1.jar:/app/impala/fe/target/dependency/aggdesigner-algorithm-6.0.jar:/app/impala/fe/target/dependency/aircompressor-0.15.jar:/app/impala/fe/target/dependency/annotations-17.0.0.jar:/app/impala/fe/target/dependency/antlr-2.7.7.jar:/app/impala/fe/target/dependency/antlr-runtime-3.3.jar:/app/impala/fe/target/dependency/aopalliance-1.0.jar:/app/impala/fe/target/dependency/aopalliance-repackaged-2.5.0-b32.jar:/app/impala/fe/target/dependency/apacheds-jdbm1-2.0.0-M2.jar:/app/impala/fe/target/dependency/asm-5.0.4.jar:/app/impala/fe/target/dependency/async-1.4.1.jar:/app/impala/fe/target/dependency/audience-annotations-0.5.0.jar:/app/impala/fe/target/dependency/avro-1.8.2-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/aws-java-sdk-bundle-1.11.375.jar:/app/impala/fe/target/dependency/azure-data-lake-store-sdk-2.2.9.jar:/app/impala/fe/target/dependency/azure-keyvault-core-0.8.0.jar:/app/impala/fe/target/dependency/azure-storage-5.4.0.jar:/app/impala/fe/target/dependency/bcpkix-jdk15on-1.60.jar:/app/impala/fe/target/dependency/bcprov-jdk15on-1.59.jar:/app/impala/fe/target/dependency/bonecp-0.8.0.RELEASE.jar:/app/impala/fe/target/dependency/calcite-core-1.12.0.jar:/app/impala/fe/target/dependency/calcite-druid-1.12.0.jar:/app/impala/fe/target/dependency/calcite-linq4j-1.12.0.jar:/app/impala/fe/target/dependency/commonj.sdo-2.1.1.jar:/app/impala/fe/target/dependency/commons-beanutils-1.9.4.jar:/app/impala/fe/target/dependency/commons-cli-1.2.jar:/app/impala/fe/target/dependency/commons-codec-1.11.jar:/app/impala/fe/target/dependency/commons-collections-3.2.2.jar:/app/impala/fe/target/dependency/commons-compiler-2.7.6.jar:/app/impala/fe/target/dependency/commons-compress-1.19.jar:/app/impala/fe/target/dependency/commons-configuration2-2.1.1.jar:/app/impala/fe/target/dependency/commons-crypto-1.0.0.jar:/app/impala/fe/target/dependency/commons-daemon-1.0.13.jar:/app/impala/fe/target/dependency/commons-dbcp-1.4.jar:/app/impala/fe/target/dependency/commons-el-1.0.jar:/app/impala/fe/target/dependency/commons-io-2.6.jar:/app/impala/fe/target/dependency/commons-lang-2.6.jar:/app/impala/fe/target/dependency/commons-lang3-3.7.jar:/app/impala/fe/target/dependency/commons-logging-1.1.3.jar:/app/impala/fe/target/dependency/commons-math-2.2.jar:/app/impala/fe/target/dependency/commons-math3-3.1.1.jar:/app/impala/fe/target/dependency/commons-net-3.1.jar:/app/impala/fe/target/dependency/commons-pool-1.5.4.jar:/app/impala/fe/target/dependency/commons-pool2-2.4.2.jar:/app/impala/fe/target/dependency/curator-client-2.12.0.jar:/app/impala/fe/target/dependency/curator-framework-2.12.0.jar:/app/impala/fe/target/dependency/curator-recipes-2.12.0.jar:/app/impala/fe/target/dependency/curator-test-2.11.1.jar:/app/impala/fe/target/dependency/curator-x-discovery-2.11.1.jar:/app/impala/fe/target/dependency/datanucleus-api-jdo-4.2.5.jar:/app/impala/fe/target/dependency/datanucleus-core-4.1.17.jar:/app/impala/fe/target/dependency/datanucleus-rdbms-4.1.17.jar:/app/impala/fe/target/dependency/derby-10.14.2.0.jar:/app/impala/fe/target/dependency/disruptor-3.3.6.jar:/app/impala/fe/target/dependency/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar:/app/impala/fe/target/dependency/eclipselink-2.5.2.jar:/app/impala/fe/target/dependency/ehcache-3.3.1.jar:/app/impala/fe/target/dependency/error_prone_annotations-2.3.1.jar:/app/impala/fe/target/dependency/fastutil-6.5.6.jar:/app/impala/fe/target/dependency/findbugs-annotations-1.3.9-1.jar:/app/impala/fe/target/dependency/flatbuffers-java-1.6.0.1.jar:/app/impala/fe/target/dependency/fluent-hc-4.3.2.jar:/app/impala/fe/target/dependency/fst-2.50.jar:/app/impala/fe/target/dependency/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/app/impala/fe/target/dependency/gethostname4j-0.0.2.jar:/app/impala/fe/target/dependency/groovy-all-2.4.11.jar:/app/impala/fe/target/dependency/gson-2.2.4.jar:/app/impala/fe/target/dependency/guava-14.0.1.jar:/app/impala/fe/target/dependency/guice-4.0.jar:/app/impala/fe/target/dependency/guice-assistedinject-3.0.jar:/app/impala/fe/target/dependency/guice-servlet-4.0.jar:/app/impala/fe/target/dependency/hadoop-annotations-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-auth-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-aws-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-azure-3.0.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hadoop-azure-datalake-3.0.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hadoop-client-api-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-client-runtime-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-common-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-distcp-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-hdfs-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-hdfs-client-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-lzo-0.4.20-SNAPSHOT.jar:/app/impala/fe/target/dependency/hadoop-mapreduce-client-app-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-mapreduce-client-common-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-mapreduce-client-core-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-yarn-api-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-yarn-client-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-yarn-common-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-yarn-server-common-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-yarn-server-resourcemanager-3.1.2.jar:/app/impala/fe/target/dependency/hadoop-yarn-server-web-proxy-3.1.2.jar:/app/impala/fe/target/dependency/hbase-client-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-common-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-hadoop2-compat-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-hadoop2-compat-2.1.0-cdh6.x-SNAPSHOT-tests.jar:/app/impala/fe/target/dependency/hbase-hadoop-compat-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-http-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-mapreduce-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-metrics-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-metrics-api-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-procedure-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-protocol-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-protocol-shaded-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-replication-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-server-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/hbase-shaded-miscellaneous-2.2.1.jar:/app/impala/fe/target/dependency/hbase-shaded-netty-2.2.1.jar:/app/impala/fe/target/dependency/hbase-shaded-protobuf-2.2.1.jar:/app/impala/fe/target/dependency/hbase-zookeeper-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/HikariCP-2.6.1.jar:/app/impala/fe/target/dependency/HikariCP-java7-2.4.12.jar:/app/impala/fe/target/dependency/hive-common-2.3.7.jar:/app/impala/fe/target/dependency/hive-exec-2.3.7.jar:/app/impala/fe/target/dependency/hive-hbase-handler-2.3.7.jar:/app/impala/fe/target/dependency/hive-llap-client-2.3.7.jar:/app/impala/fe/target/dependency/hive-llap-common-2.3.7.jar:/app/impala/fe/target/dependency/hive-llap-tez-2.3.7.jar:/app/impala/fe/target/dependency/hive-metastore-2.3.7.jar:/app/impala/fe/target/dependency/hive-orc-2.2.0.jar:/app/impala/fe/target/dependency/hive-serde-2.3.7.jar:/app/impala/fe/target/dependency/hive-service-2.3.7.jar:/app/impala/fe/target/dependency/hive-service-rpc-2.3.7.jar:/app/impala/fe/target/dependency/hive-shims-0.23-2.3.7.jar:/app/impala/fe/target/dependency/hive-shims-2.3.7.jar:/app/impala/fe/target/dependency/hive-shims-common-2.3.7.jar:/app/impala/fe/target/dependency/hive-shims-scheduler-2.3.7.jar:/app/impala/fe/target/dependency/hive-storage-api-2.4.0.jar:/app/impala/fe/target/dependency/hk2-api-2.5.0-b32.jar:/app/impala/fe/target/dependency/hk2-locator-2.5.0-b32.jar:/app/impala/fe/target/dependency/hk2-utils-2.5.0-b32.jar:/app/impala/fe/target/dependency/htrace-core4-4.1.0-incubating.jar:/app/impala/fe/target/dependency/httpclient-4.5.3.jar:/app/impala/fe/target/dependency/httpcore-4.4.9.jar:/app/impala/fe/target/dependency/httpmime-4.5.3.jar:/app/impala/fe/target/dependency/hudi-common-0.5.0-incubating.jar:/app/impala/fe/target/dependency/hudi-hadoop-mr-0.5.0-incubating.jar:/app/impala/fe/target/dependency/impala-data-source-api-1.0-SNAPSHOT.jar:/app/impala/fe/target/dependency/ivy-2.4.0.jar:/app/impala/fe/target/dependency/jackson-annotations-2.9.10.jar:/app/impala/fe/target/dependency/jackson-core-2.10.0.jar:/app/impala/fe/target/dependency/jackson-core-asl-1.9.13.jar:/app/impala/fe/target/dependency/jackson-databind-2.10.0.jar:/app/impala/fe/target/dependency/jackson-jaxrs-1.9.13.jar:/app/impala/fe/target/dependency/jackson-jaxrs-base-2.9.10.jar:/app/impala/fe/target/dependency/jackson-jaxrs-json-provider-2.9.10.jar:/app/impala/fe/target/dependency/jackson-mapper-asl-1.9.13-cloudera.1.jar:/app/impala/fe/target/dependency/jackson-module-jaxb-annotations-2.9.10.jar:/app/impala/fe/target/dependency/jackson-xc-1.9.2.jar:/app/impala/fe/target/dependency/jamon-runtime-2.3.1.jar:/app/impala/fe/target/dependency/janino-2.7.6.jar:/app/impala/fe/target/dependency/jasper-compiler-5.5.23.jar:/app/impala/fe/target/dependency/jasper-runtime-5.5.23.jar:/app/impala/fe/target/dependency/java-cup-0.11-a-czt02-cdh.jar:/app/impala/fe/target/dependency/java-cup-runtime-0.11-a-czt01-cdh.jar:/app/impala/fe/target/dependency/javassist-3.18.1-GA.jar:/app/impala/fe/target/dependency/java-util-1.9.0.jar:/app/impala/fe/target/dependency/javax.activation-api-1.2.0.jar:/app/impala/fe/target/dependency/javax.annotation-api-1.2.jar:/app/impala/fe/target/dependency/javax.el-3.0.1-b08.jar:/app/impala/fe/target/dependency/javax.inject-1.jar:/app/impala/fe/target/dependency/javax.inject-2.5.0-b32.jar:/app/impala/fe/target/dependency/javax.jdo-3.2.0-m3.jar:/app/impala/fe/target/dependency/javax.json-1.0.2.jar:/app/impala/fe/target/dependency/javax.persistence-2.1.0.jar:/app/impala/fe/target/dependency/javax.servlet-api-3.1.0.jar:/app/impala/fe/target/dependency/javax.servlet.jsp-2.3.2.jar:/app/impala/fe/target/dependency/javax.servlet.jsp-api-2.3.1.jar:/app/impala/fe/target/dependency/javax.ws.rs-api-2.0.1.jar:/app/impala/fe/target/dependency/javolution-5.5.1.jar:/app/impala/fe/target/dependency/jaxb-api-2.2.11.jar:/app/impala/fe/target/dependency/jaxb-impl-2.2.3-1.jar:/app/impala/fe/target/dependency/jcip-annotations-1.0-1.jar:/app/impala/fe/target/dependency/jcl-over-slf4j-1.7.24.jar:/app/impala/fe/target/dependency/jcodings-1.0.18.jar:/app/impala/fe/target/dependency/jdo-api-3.0.1.jar:/app/impala/fe/target/dependency/jersey-client-1.19.jar:/app/impala/fe/target/dependency/jersey-client-2.25.1.jar:/app/impala/fe/target/dependency/jersey-common-2.25.1.jar:/app/impala/fe/target/dependency/jersey-container-servlet-core-2.25.1.jar:/app/impala/fe/target/dependency/jersey-core-1.19.jar:/app/impala/fe/target/dependency/jersey-guava-2.25.1.jar:/app/impala/fe/target/dependency/jersey-guice-1.19.jar:/app/impala/fe/target/dependency/jersey-json-1.19.jar:/app/impala/fe/target/dependency/jersey-media-jaxb-2.25.1.jar:/app/impala/fe/target/dependency/jersey-server-1.19.jar:/app/impala/fe/target/dependency/jersey-server-2.25.1.jar:/app/impala/fe/target/dependency/jersey-servlet-1.19.jar:/app/impala/fe/target/dependency/jettison-1.1.jar:/app/impala/fe/target/dependency/jetty-util-9.3.25.v20180904.jar:/app/impala/fe/target/dependency/jline-2.12.jar:/app/impala/fe/target/dependency/jna-5.2.0.jar:/app/impala/fe/target/dependency/jna-platform-5.2.0.jar:/app/impala/fe/target/dependency/joda-time-2.9.9.jar:/app/impala/fe/target/dependency/joni-2.1.11.jar:/app/impala/fe/target/dependency/jpam-1.1.jar:/app/impala/fe/target/dependency/jsch-0.1.54.jar:/app/impala/fe/target/dependency/json-20090211.jar:/app/impala/fe/target/dependency/json-io-2.5.1.jar:/app/impala/fe/target/dependency/json-simple-1.1.1.jar:/app/impala/fe/target/dependency/json-smart-2.3.jar:/app/impala/fe/target/dependency/jsp-api-2.0.jar:/app/impala/fe/target/dependency/jsp-api-2.1.jar:/app/impala/fe/target/dependency/jsr305-3.0.0.jar:/app/impala/fe/target/dependency/jsr311-api-1.1.1.jar:/app/impala/fe/target/dependency/jta-1.1.jar:/app/impala/fe/target/dependency/kerb-admin-1.0.0.jar:/app/impala/fe/target/dependency/kerb-client-1.0.0.jar:/app/impala/fe/target/dependency/kerb-common-1.0.0.jar:/app/impala/fe/target/dependency/kerb-core-1.0.0.jar:/app/impala/fe/target/dependency/kerb-crypto-1.0.0.jar:/app/impala/fe/target/dependency/kerb-identity-1.0.0.jar:/app/impala/fe/target/dependency/kerb-server-1.0.0.jar:/app/impala/fe/target/dependency/kerb-simplekdc-1.0.0.jar:/app/impala/fe/target/dependency/kerb-util-1.0.0.jar:/app/impala/fe/target/dependency/kerby-asn1-1.0.0.jar:/app/impala/fe/target/dependency/kerby-config-1.0.0.jar:/app/impala/fe/target/dependency/kerby-pkix-1.0.0.jar:/app/impala/fe/target/dependency/kerby-util-1.0.0.jar:/app/impala/fe/target/dependency/kerby-xdr-1.0.0.jar:/app/impala/fe/target/dependency/kryo-shaded-4.0.2.jar:/app/impala/fe/target/dependency/kudu-client-1.12.0.jar:/app/impala/fe/target/dependency/libfb303-0.9.3.jar:/app/impala/fe/target/dependency/libthrift-0.9.3-1.jar:/app/impala/fe/target/dependency/log4j-1.2.17.jar:/app/impala/fe/target/dependency/log4j-api-2.8.2.jar:/app/impala/fe/target/dependency/log4j-core-2.8.2.jar:/app/impala/fe/target/dependency/log4j-web-2.8.2.jar:/app/impala/fe/target/dependency/logredactor-2.0.7.jar:/app/impala/fe/target/dependency/mail-1.4.jar:/app/impala/fe/target/dependency/metrics-core-3.2.2.jar:/app/impala/fe/target/dependency/metrics-json-3.1.0.jar:/app/impala/fe/target/dependency/metrics-jvm-3.1.0.jar:/app/impala/fe/target/dependency/minlog-1.3.0.jar:/app/impala/fe/target/dependency/mssql-jdbc-6.2.1.jre7.jar:/app/impala/fe/target/dependency/mysql-connector-java-8.0.19.jar:/app/impala/fe/target/dependency/netty-3.10.6.Final.jar:/app/impala/fe/target/dependency/netty-all-4.1.42.Final.jar:/app/impala/fe/target/dependency/nimbus-jose-jwt-4.41.1.jar:/app/impala/fe/target/dependency/noggit-0.8.jar:/app/impala/fe/target/dependency/objenesis-2.6.jar:/app/impala/fe/target/dependency/okhttp-2.7.5.jar:/app/impala/fe/target/dependency/okio-1.6.0.jar:/app/impala/fe/target/dependency/opencsv-2.3.jar:/app/impala/fe/target/dependency/orc-core-1.6.2.jar:/app/impala/fe/target/dependency/orc-shims-1.6.2.jar:/app/impala/fe/target/dependency/oro-2.0.8.jar:/app/impala/fe/target/dependency/osgi-resource-locator-1.0.1.jar:/app/impala/fe/target/dependency/paranamer-2.8.jar:/app/impala/fe/target/dependency/parquet-hadoop-bundle-1.10.99-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/postgresql-42.2.5.jar:/app/impala/fe/target/dependency/protobuf-java-2.5.0.jar:/app/impala/fe/target/dependency/query-event-hook-api-1.0-SNAPSHOT.jar:/app/impala/fe/target/dependency/ranger-plugins-audit-2.0.0.jar:/app/impala/fe/target/dependency/ranger-plugins-common-2.0.0.jar:/app/impala/fe/target/dependency/ranger-plugins-cred-2.0.0.jar:/app/impala/fe/target/dependency/re2j-1.1.jar:/app/impala/fe/target/dependency/rocksdbjni-5.17.2.jar:/app/impala/fe/target/dependency/sentry-binding-hive-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-binding-hive-common-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-binding-hive-conf-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-binding-hive-follower-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-core-common-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-core-model-db-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-core-model-indexer-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-core-model-kafka-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-core-model-solr-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-hdfs-common-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-policy-common-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-policy-engine-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-provider-cache-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-provider-common-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-provider-db-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-provider-file-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-service-api-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-service-client-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/sentry-shaded-miscellaneous-2.1.0-cdh6.x-SNAPSHOT.jar:/app/impala/fe/target/dependency/shiro-cache-1.4.0.jar:/app/impala/fe/target/dependency/shiro-config-core-1.4.0.jar:/app/impala/fe/target/dependency/shiro-config-ogdl-1.4.0.jar:/app/impala/fe/target/dependency/shiro-core-1.4.0.jar:/app/impala/fe/target/dependency/shiro-crypto-cipher-1.4.0.jar:/app/impala/fe/target/dependency/shiro-crypto-core-1.4.0.jar:/app/impala/fe/target/dependency/shiro-crypto-hash-1.4.0.jar:/app/impala/fe/target/dependency/shiro-event-1.4.0.jar:/app/impala/fe/target/dependency/shiro-lang-1.4.0.jar:/app/impala/fe/target/dependency/sizeof-0.3.0.jar:/app/impala/fe/target/dependency/slf4j-api-1.7.25.jar:/app/impala/fe/target/dependency/slf4j-log4j12-1.7.25.jar:/app/impala/fe/target/dependency/snappy-0.2.jar:/app/impala/fe/target/dependency/snappy-java-1.1.4.jar:/app/impala/fe/target/dependency/solr-solrj-7.7.1.jar:/app/impala/fe/target/dependency/ST4-4.0.4.jar:/app/impala/fe/target/dependency/stax2-api-3.1.4.jar:/app/impala/fe/target/dependency/stax-api-1.0.1.jar:/app/impala/fe/target/dependency/stringtemplate-3.2.1.jar:/app/impala/fe/target/dependency/tephra-api-0.6.0.jar:/app/impala/fe/target/dependency/tephra-core-0.6.0.jar:/app/impala/fe/target/dependency/tephra-hbase-compat-1.0-0.6.0.jar:/app/impala/fe/target/dependency/twill-api-0.6.0-incubating.jar:/app/impala/fe/target/dependency/twill-common-0.6.0-incubating.jar:/app/impala/fe/target/dependency/twill-core-0.6.0-incubating.jar:/app/impala/fe/target/dependency/twill-discovery-api-0.6.0-incubating.jar:/app/impala/fe/target/dependency/twill-discovery-core-0.6.0-incubating.jar:/app/impala/fe/target/dependency/twill-zookeeper-0.6.0-incubating.jar:/app/impala/fe/target/dependency/validation-api-1.1.0.Final.jar:/app/impala/fe/target/dependency/velocity-1.5.jar:/app/impala/fe/target/dependency/wildfly-openssl-1.0.4.Final.jar:/app/impala/fe/target/dependency/woodstox-core-5.0.3.jar:/app/impala/fe/target/dependency/woodstox-core-asl-4.4.1.jar:/app/impala/fe/target/dependency/xz-1.8.jar:/app/impala/fe/target/dependency/yarn-extras-0.1-SNAPSHOT.jar:/app/impala/fe/target/dependency/zookeeper-3.4.5-cdh6.x-SNAPSHOT.jar -
拷贝mysql连接jar
可以加上上述classpath里
-
启动各个进程
export CLASSPATH=/app/impala/../hadoop-lzo/build/hadoop-lzo-0.4.15.jar:/app/impala/fe/src/test/resources:/app/impala/fe/target/classes:/app/impala/fe/target/dependency:.:/usr/java/jdk1.8.0_201-amd64/lib/dt.jar:/usr/java/jdk1.8.0_201-amd64/lib/tools.jar:/app/impala/fe/target/dependency/*.jar source /app/impala/config.sh /app/impala/be/build/debug/service/statestored ${IMPALA_STATE_STORE_ARGS} /app/impala/be/build/debug/service/catalogd ${IMPALA_CATALOG_ARGS} #注意指令有变化 # /app/impala/be/build/debug/service/impalad -log_dir=/app/impala/logs/impala -state_store_host=sit1 -catalog_service_host=sit1 -state_store_port=24000 -be_port=22000 /app/impala/be/build/debug/service/impalad ${IMPALA_SERVER_ARGS} -
测试impala
/app/impala/shell/build/impala-shell-3.4.0-RELEASE/impala-shell
问题解决
-
报错
Exception thrown when executing query : SELECT ‘org.apache.hadoop.hive.metastore.model.MVersionTable’ AS
NUCLEUS_TYPE,A0.SCHEMA_VERSION,A1.SCHEMA_VERSION,A0.VERSION_COMMENT,A0.VER_IDFROMVERSIONA0INNER JOINCDH_VERSIONA1ONA0.VER_ID=A1.VER_ID这是由于impala默认的hive和hadoop版本是CDH的，所以需要更换上面的impala的相关依赖替换上面的hadoop、hive、habase开头的相关包
-
报错
Event: 0.319 loading class org/apache/hadoop/hdfs/web/SWebHdfsFileSystem Event: 0.319 loading class org/apache/hadoop/hdfs/web/SWebHdfsFileSystem done Event: 0.329 loading class org/apache/hadoop/hive/ql/io/NullScanFileSystem Event: 0.329 loading class org/apache/hadoop/hive/ql/io/NullScanFileSystem done Event: 0.330 loading class org/apache/hadoop/security/token/Token Event: 0.330 loading class org/apache/hadoop/security/token/Token done Event: 0.332 loading class org/apache/hadoop/hive/ql/io/ProxyLocalFileSystem Event: 0.332 loading class org/apache/hadoop/hive/ql/io/ProxyLocalFileSystem done Event: 0.333 loading class org/apache/hadoop/shaded/com/ctc/wstx/io/InputBootstrapper Event: 0.333 loading class org/apache/hadoop/shaded/com/ctc/wstx/io/InputBootstrapper done这是由于hadoop3.1.2除了需要impala那些个同名jar文件外还需要引入hadoop-client-runtime-3.1.2.jar
-
报错
E0312 14:10:09.454614 30004 impala-server.cc:371] Currently configured default filesystem: ProxyLocalFileSystem. fs.defaultFS (file:///) is not supported.
这个问题太恶心了，网上几乎没有解答对的，这个问题的解决办法就是，当你启动impala的时候，当前目录必须是hadoop配置文件的目录，要不就报上面那个错误，不知道为什么指定了hadoop的目录不可以
-
查询报错
CAUSED BY: NoClassDefFoundError: org/apache/hadoop/fs/adl/AdlFileSystem
1561
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
