Impala编译错误及解决办法汇总

已于 2023-02-23 22:02:19 修改
阅读量993 收藏
点赞数 1
分类专栏： Impala 文章标签： hadoop
于 2023-02-23 21:42:30 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/huang_quanlong/article/details/109839490
版权

Impala编译错误及解决办法汇总

Impala repo里有C++、Java、Python、Bash的代码，编译过程中可能遇到各种问题。这个文章将作为一个汇总，以后每遇到一个编译问题，解决完后都会更新一下。

FE编译问题

Maven报错：Return code is: 501, ReasonPhrase:HTTPS Required.

在一台CentOS 7.4机器上编译Impala 4.0时遇到错误：

$ (pushd java && mvn clean package -DskipTests)
......
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Impala Parent POM 4.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.4.1/maven-clean-plugin-2.4.1.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Impala Parent POM .......................... FAILURE [0.281s]
[INFO] Test Data Generator ............................... SKIPPED
[INFO] impala-minimal-s3a-aws-sdk ........................ SKIPPED
[INFO] impala-minimal-hive-exec .......................... SKIPPED
[INFO] Impala Executor Dependencies ...................... SKIPPED
[INFO] Apache Impala External Data Source ................ SKIPPED
[INFO] Apache Impala External Data Source API ............ SKIPPED
[INFO] Apache Impala External Data Source Sample ......... SKIPPED
[INFO] Apache Impala External Data Source Test Library ... SKIPPED
[INFO] Apache Impala Query Event Hook API ................ SKIPPED
[INFO] YARN Extras ....................................... SKIPPED
[INFO] Apache Impala Query Engine Frontend ............... SKIPPED
[INFO] Impala Nested Table Flattener ..................... SKIPPED
[INFO] test-hive-udfs .................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.408s
[INFO] Finished at: Thu Nov 19 17:35:58 PST 2020
[INFO] Final Memory: 16M/951M
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.apache.maven.plugins:maven-clean-plugin:2.4.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-clean-plugin:jar:2.4.1: Could not transfer artifact org.apache.maven.plugins:maven-clean-plugin:pom:2.4.1 from/to central (http://repo.maven.apache.org/maven2): Failed to transfer file: http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.4.1/maven-clean-plugin-2.4.1.pom. Return code is: 501, ReasonPhrase:HTTPS Required. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

这个跟Impala 4.0无法，主要是maven repo要求用HTTPS的方式访问。而这台机器上的maven版本比较旧，使用的还是HTTP的URL，自己在 ~/.m2/settings.xml 里写个 HTTPS 的 URL 就行了：

<?xml version="1.0" encoding="UTF-8"?>
<settings>
    <mirrors>
        <mirror>
            <id>maven</id>
            <name>central maven</name>
            <url>https://repo.maven.apache.org/maven2</url>
            <mirrorOf>central</mirrorOf>
        </mirror>
    </mirrors>
</settings>

国内的服务器使用阿里云之类的镜像会更快，也没有HTTPS的强制限制，可以把 ~/.m2/settings.xml 改为：

<?xml version="1.0" encoding="UTF-8"?>
<settings>
    <mirrors>
        <mirror>
            <id>alimaven</id>
            <name>aliyun maven</name>
            <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
            <mirrorOf>central</mirrorOf>
        </mirror>
    </mirrors>
</settings>

Maven报错：net.sourceforge.czt.dev:cup-maven-plugin:1.6-cdh

[INFO] BUILD FAILURE
[ERROR] Plugin net.sourceforge.czt.dev:cup-maven-plugin:1.6-cdh or one of its dependencies could not be resolved: Failed to read artifact descriptor for net.sourceforge.czt.dev:cup-maven-plugin:jar:1.6-cdh: Could not find artifact net.sourceforge.czt.dev:cup-maven-plugin:pom:1.6-cdh in cloudera.thirdparty.repo (https://repository.cloudera.com/content/repositories/third-party) -> [Help 1]

diff --git a/impala-parent/pom.xml b/impala-parent/pom.xml
index 494749785..28b19128f 100644
--- a/impala-parent/pom.xml
+++ b/impala-parent/pom.xml
@@ -146,7 +146,7 @@ under the License.
     </repository>
     <repository>
       <id>cdh.rcs.releases.repo</id>
-      <url>https://repository.cloudera.com/content/groups/cdh-releases-rcs</url>
+      <url>https://repository.cloudera.com/artifactory/cdh-releases-rcs</url>
       <name>CDH Releases Repository</name>
       <snapshots>
         <enabled>true</enabled>
@@ -197,7 +197,7 @@ under the License.
     </repository>
     <repository>
       <id>cloudera.thirdparty.repo</id>
-      <url>https://repository.cloudera.com/content/repositories/third-party</url>
+      <url>https://repository.cloudera.com/artifactory/cdh-releases-rcs</url>
       <name>Cloudera Third Party Repository</name>
       <snapshots>
         <enabled>false</enabled>
@@ -236,7 +236,7 @@ under the License.
   <pluginRepositories>
     <pluginRepository>
       <id>cloudera.thirdparty.repo</id>
-      <url>https://repository.cloudera.com/content/repositories/third-party</url>
+      <url>https://repository.cloudera.com/artifactory/cdh-releases-rcs</url>
       <name>Cloudera Third Party Repository</name>
       <snapshots>
         <enabled>false</enabled>

BE编译问题

从make切换到ninja报undefined reference错误

Makefile和Ninja是两个不同的编译工具，使用了一个之后，没法立马切换到另一个做增量编译。如果需要从make切换到ninja（反方向同理），初次使用buildall.sh时不要加 -noclean 选项，否则会报如下错误：

[100%] Built target loggingsupport
/home/quanlong/workspace/Impala/be/src/kudu/util/mutex.h:120: error: undefined reference to 'kudu::Mutex::AssertAcquired() const'
../../build/debug/runtime/libRuntime.a(types.cc.o):types.cc:llvm::VerifyDisableABIBreakingChecks: error: undefined reference to 'llvm::DisableABIBreakingChecks'
/home/quanlong/workspace/Impala/be/src/udf/udf.cc:154: error: undefined reference to 'fLI::FLAGS_stress_fn_ctx_alloc'
/home/quanlong/workspace/Impala/be/src/udf/udf.cc:155: error: undefined reference to 'fLI::FLAGS_stress_fn_ctx_alloc'
/home/quanlong/workspace/Impala/be/src/util/thread.cc:304: error: undefined reference to 'fLB::FLAGS_thread_creation_fault_injection'
/home/quanlong/workspace/Impala/be/src/catalog/catalog-server.cc:293: error: undefined reference to 'fLI::FLAGS_stress_catalog_startup_delay_ms'
/home/quanlong/workspace/Impala/be/src/catalog/catalog-server.cc:294: error: undefined reference to 'fLI::FLAGS_stress_catalog_startup_delay_ms'
../../build/debug/codegen/libCodeGen.a(instruction-counter.cc.o):instruction-counter.cc:llvm::VerifyDisableABIBreakingChecks: error: undefined reference to 'llvm::DisableABIBreakingChecks'
../../build/debug/codegen/libCodeGen.a(codegen-callgraph.cc.o):codegen-callgraph.cc:llvm::VerifyDisableABIBreakingChecks: error: undefined reference to 'llvm::DisableABIBreakingChecks'
../../build/debug/codegen/libCodeGen.a(codegen-symbol-emitter.cc.o):codegen-symbol-emitter.cc:llvm::VerifyDisableABIBreakingChecks: error: undefined reference to 'llvm::DisableABIBreakingChecks'
/home/quanlong/workspace/Impala/be/src/exec/exchange-node.cc:97: error: undefined reference to 'fLI::FLAGS_stress_datastream_recvr_delay_ms'
/home/quanlong/workspace/Impala/be/src/exec/exchange-node.cc:98: error: undefined reference to 'fLI::FLAGS_stress_datastream_recvr_delay_ms'
/home/quanlong/workspace/Impala/be/src/exec/hdfs-scan-node-base.cc:706: error: undefined reference to 'fLB::FLAGS_skip_file_runtime_filtering'
/home/quanlong/workspace/Impala/be/src/runtime/io/disk-io-mgr.cc:798: error: undefined reference to 'fLI::FLAGS_stress_scratch_write_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/disk-io-mgr.cc:799: error: undefined reference to 'fLI::FLAGS_stress_scratch_write_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/hdfs-file-reader.cc:114: error: undefined reference to 'fLI::FLAGS_stress_disk_read_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/hdfs-file-reader.cc:115: error: undefined reference to 'fLI::FLAGS_stress_disk_read_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/local-file-reader.cc:60: error: undefined reference to 'fLI::FLAGS_stress_disk_read_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/local-file-reader.cc:61: error: undefined reference to 'fLI::FLAGS_stress_disk_read_delay_ms'
/home/quanlong/workspace/Impala/be/src/kudu/rpc/messenger.cc:261: error: undefined reference to 'kudu::ThreadRestrictions::AssertWaitAllowed()'
/home/quanlong/workspace/Impala/be/src/kudu/util/thread_restrictions.h:77: error: undefined reference to 'kudu::ThreadRestrictions::SetWaitAllowed(bool)'
/home/quanlong/workspace/Impala/be/src/kudu/util/thread_restrictions.h:78: error: undefined reference to 'kudu::ThreadRestrictions::SetWaitAllowed(bool)'
/home/quanlong/workspace/Impala/be/src/kudu/util/countdown_latch.h:71: error: undefined reference to 'kudu::ThreadRestrictions::AssertWaitAllowed()'
/home/quanlong/workspace/Impala/be/src/kudu/rpc/reactor.cc:494: error: undefined reference to 'kudu::ThreadRestrictions::SetWaitAllowed(bool)'
/home/quanlong/workspace/Impala/be/src/kudu/rpc/reactor.cc:495: error: undefined reference to 'kudu::ThreadRestrictions::SetIOAllowed(bool)'
/home/quanlong/workspace/Impala/be/src/kudu/util/countdown_latch.h:81: error: undefined reference to 'kudu::ThreadRestrictions::AssertWaitAllowed()'
../../../toolchain/toolchain-packages-gcc7.5.0/openldap-2.4.47/lib/libldap.a(os-ip.o):os-ip.c:function ldap_int_poll: warning: `sys_nerr' is deprecated; use `strerror' or `strerror_r' instead
../../../toolchain/toolchain-packages-gcc7.5.0/openldap-2.4.47/lib/libldap.a(os-ip.o):os-ip.c:function ldap_int_poll: warning: `sys_errlist' is deprecated; use `strerror' or `strerror_r' instead
collect2: error: ld returned 1 exit status
../../../toolchain/toolchain-packages-gcc7.5.0/openldap-2.4.47/lib/libldap.a(os-ip.o):os-ip.c:function ldap_int_poll: warning: `sys_nerr' is deprecated; use `strerror' or `strerror_r' instead
../../../toolchain/toolchain-packages-gcc7.5.0/openldap-2.4.47/lib/libldap.a(os-ip.o):os-ip.c:function ldap_int_poll: warning: `sys_errlist' is deprecated; use `strerror' or `strerror_r' instead
Generated: /home/quanlong/workspace/Impala/logs/extra_junit_xml_logs/generate_junitxml.build.5414ebd990c149b6136b79ddf15467ad.20210922_13_40_49.xml
[100%] Built target fesupport
be/src/util/CMakeFiles/impala-profile-tool.dir/build.make:212: recipe for target 'be/build/debug/util/impala-profile-tool' failed
make[3]: *** [be/build/debug/util/impala-profile-tool] Error 1
CMakeFiles/Makefile2:11043: recipe for target 'be/src/util/CMakeFiles/impala-profile-tool.dir/all' failed
make[2]: *** [be/src/util/CMakeFiles/impala-profile-tool.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
/home/quanlong/workspace/Impala/be/src/catalog/catalog-server.cc:293: error: undefined reference to 'fLI::FLAGS_stress_catalog_startup_delay_ms'
/home/quanlong/workspace/Impala/be/src/catalog/catalog-server.cc:294: error: undefined reference to 'fLI::FLAGS_stress_catalog_startup_delay_ms'
/home/quanlong/workspace/Impala/be/src/kudu/util/mutex.h:120: error: undefined reference to 'kudu::Mutex::AssertAcquired() const'
../../build/debug/runtime/libRuntime.a(types.cc.o):types.cc:llvm::VerifyDisableABIBreakingChecks: error: undefined reference to 'llvm::DisableABIBreakingChecks'
/home/quanlong/workspace/Impala/be/src/udf/udf.cc:154: error: undefined reference to 'fLI::FLAGS_stress_fn_ctx_alloc'
/home/quanlong/workspace/Impala/be/src/udf/udf.cc:155: error: undefined reference to 'fLI::FLAGS_stress_fn_ctx_alloc'
/home/quanlong/workspace/Impala/be/src/util/thread.cc:304: error: undefined reference to 'fLB::FLAGS_thread_creation_fault_injection'
../../build/debug/codegen/libCodeGen.a(instruction-counter.cc.o):instruction-counter.cc:llvm::VerifyDisableABIBreakingChecks: error: undefined reference to 'llvm::DisableABIBreakingChecks'
../../build/debug/codegen/libCodeGen.a(codegen-callgraph.cc.o):codegen-callgraph.cc:llvm::VerifyDisableABIBreakingChecks: error: undefined reference to 'llvm::DisableABIBreakingChecks'
../../build/debug/codegen/libCodeGen.a(codegen-symbol-emitter.cc.o):codegen-symbol-emitter.cc:llvm::VerifyDisableABIBreakingChecks: error: undefined reference to 'llvm::DisableABIBreakingChecks'
/home/quanlong/workspace/Impala/be/src/exec/exchange-node.cc:97: error: undefined reference to 'fLI::FLAGS_stress_datastream_recvr_delay_ms'
/home/quanlong/workspace/Impala/be/src/exec/exchange-node.cc:98: error: undefined reference to 'fLI::FLAGS_stress_datastream_recvr_delay_ms'
/home/quanlong/workspace/Impala/be/src/exec/hdfs-scan-node-base.cc:706: error: undefined reference to 'fLB::FLAGS_skip_file_runtime_filtering'
/home/quanlong/workspace/Impala/be/src/runtime/io/disk-io-mgr.cc:798: error: undefined reference to 'fLI::FLAGS_stress_scratch_write_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/disk-io-mgr.cc:799: error: undefined reference to 'fLI::FLAGS_stress_scratch_write_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/hdfs-file-reader.cc:114: error: undefined reference to 'fLI::FLAGS_stress_disk_read_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/hdfs-file-reader.cc:115: error: undefined reference to 'fLI::FLAGS_stress_disk_read_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/local-file-reader.cc:60: error: undefined reference to 'fLI::FLAGS_stress_disk_read_delay_ms'
/home/quanlong/workspace/Impala/be/src/runtime/io/local-file-reader.cc:61: error: undefined reference to 'fLI::FLAGS_stress_disk_read_delay_ms'
/home/quanlong/workspace/Impala/be/src/kudu/rpc/messenger.cc:261: error: undefined reference to 'kudu::ThreadRestrictions::AssertWaitAllowed()'
/home/quanlong/workspace/Impala/be/src/kudu/util/thread_restrictions.h:77: error: undefined reference to 'kudu::ThreadRestrictions::SetWaitAllowed(bool)'
/home/quanlong/workspace/Impala/be/src/kudu/util/thread_restrictions.h:78: error: undefined reference to 'kudu::ThreadRestrictions::SetWaitAllowed(bool)'
/home/quanlong/workspace/Impala/be/src/kudu/util/countdown_latch.h:71: error: undefined reference to 'kudu::ThreadRestrictions::AssertWaitAllowed()'
/home/quanlong/workspace/Impala/be/src/kudu/rpc/reactor.cc:494: error: undefined reference to 'kudu::ThreadRestrictions::SetWaitAllowed(bool)'
/home/quanlong/workspace/Impala/be/src/kudu/rpc/reactor.cc:495: error: undefined reference to 'kudu::ThreadRestrictions::SetIOAllowed(bool)'
/home/quanlong/workspace/Impala/be/src/kudu/util/countdown_latch.h:81: error: undefined reference to 'kudu::ThreadRestrictions::AssertWaitAllowed()'
../../../toolchain/toolchain-packages-gcc7.5.0/openldap-2.4.47/lib/libldap.a(os-ip.o):os-ip.c:function ldap_int_poll: warning: `sys_nerr' is deprecated; use `strerror' or `strerror_r' instead
../../../toolchain/toolchain-packages-gcc7.5.0/openldap-2.4.47/lib/libldap.a(os-ip.o):os-ip.c:function ldap_int_poll: warning: `sys_errlist' is deprecated; use `strerror' or `strerror_r' instead
collect2: error: ld returned 1 exit status
Generated: /home/quanlong/workspace/Impala/logs/extra_junit_xml_logs/generate_junitxml.build.bf16cdccc74efd5090a3231aa745ac89.20210922_13_40_49.xml
be/src/service/CMakeFiles/impalad.dir/build.make:213: recipe for target 'be/build/debug/service/impalad' failed
make[3]: *** [be/build/debug/service/impalad] Error 1
CMakeFiles/Makefile2:8362: recipe for target 'be/src/service/CMakeFiles/impalad.dir/all' failed
make[2]: *** [be/src/service/CMakeFiles/impalad.dir/all] Error 2

Python问题

ImportError: No module named xxx

比如报如下的错：

Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
  File "/home/quanlong/workspace/Impala/bin/start-impala-cluster.py", line 23, in <module>
    import getpass
ImportError: No module named getpass

这是impala的virtualenv有问题，可以重新构建一下virtualenv。修改 bin/impala-python-common.sh，在最后一行加上 “-r” 参数：

diff --git a/bin/impala-python-common.sh b/bin/impala-python-common.sh
index 4026029fa..ae636316c 100644
--- a/bin/impala-python-common.sh
+++ b/bin/impala-python-common.sh
@@ -29,4 +29,4 @@ export LD_LIBRARY_PATH="$(python "$IMPALA_HOME/infra/python/bootstrap_virtualenv
 
 PY_DIR="$(dirname "$0")/../infra/python"
 PY_ENV_DIR="${PY_DIR}/env-gcc${IMPALA_GCC_VERSION}"
-python "$PY_DIR/bootstrap_virtualenv.py"
+python "$PY_DIR/bootstrap_virtualenv.py" -r

然后执行该脚本即可

$ bash bin/impala-python-common.sh 
INFO:bootstrap_virtualenv:Creating python virtualenv
INFO:bootstrap_virtualenv:Installing setuptools into the virtualenv
INFO:bootstrap_virtualenv:Installing packages into the virtualenv
INFO:bootstrap_virtualenv:Installing Kudu into the virtualenv

bin/impala-python-common.sh 其实就是 infra/python/bootstrap_virtualenv.py 的封装脚本，而这个python脚本有个 -r 参数，可以强制重建virtualenv。
注：修复完后记得把 bin/impala-python-common.sh 改回去。

stiga-huang
关注
  • 1
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
dbeaver impala jdbc连接包
01-20
dbeaver中连接impala所需jdbc包，适用于CDH5.16.2，impala2.12
Impala并发查询缓慢问题解决方案.docx
10-26
Impala并发查询缓慢问题解决方案.docx
impala自定义日期处理的udf函数
03-23
由于impala处理日期的函数如date_sub(),date_trunc(),last_day()等这些日期处理函数还需要进行日期格式化为yyyy-MM-dd使用,sql代码段过长,导致频繁嵌套过于复杂.所以自定义udf函数解决这些问题.以下为实现过程.
CentOS 7 搭建 Impala 4.1.2 + Kudu 1.15.0 测试环境
最新发布
一只努力的微服务
07-07 1209
通过 Web 页面看一下 Kudu，地址为：http://127.0.0.1:8051。打开：http://127.0.0.1:25000/backends。打开：http://127.0.0.1:25020/catalog。打开：http://127.0.0.1:25010/metrics。下载 java 安装包，需要登录 oracle，请自行下载。这部分不过于详细介绍，如果有现成环境也可以直接拿来使用。编译完成后，自行打的 RPM 包进行安装。至此，所有的安装验证就完成了。
impala 4.0.0 开发环境搭建
PeersLee的博客
06-15 3417
源码
Doris集群环境搭建
Learning_xzj的博客
10-18 4019
首页 - Apache DorisJava语言开发，存储和维护集群的元数据，负责接收和解析用户的查询请求，规划查询计化，调度查询结果。C++语言开发，Doris 系统的计算和存储节点，根据FE生成的物理执行计划，然后进行查询（分布式，多节点并行执行查询，统一汇总），同时BE还会将数据存储为多副本。BrokerBroker 为一个独立的无状态进程。封装了文件系统接口，提供 Doris 读取远端存储系统中文件的能力，包括HDFS，S3，BOS等。Doris端口列表Doris镜像版本与编译环境对应关系。
Impala手动安装
热门推荐
数据科学汇集
05-15 1万+
Impala 安装 1 准备 1.1 新增Impala用户并加入sudo权限 chmod u+w /etc/sudoers vi /etc/sudoers impala   ALL=(ALL) NOPASSWD: NOPASSWD: ALL chmodu-w /etc/sudoers 1.2准备rpm安装文件和jar包 1.2.1 见附件内的Hadoop5.0.1Impala1.3
【Linux】Impala源码编译全过程小结
qq_51228157的博客
07-26 2277
Impala编译实战
impala编译安装
sinat_26781639的博客
03-16 2369
前言 本文主要关于Impala-cdh5-2.12.0_5.16.1 源码编译与安装~ 背景 公司需要将hdfs迁移到腾讯云的chdfs。chdfs实现了hdfs的协议，可以按照数据实际内存来付费，会节省不少钱。然而在测试过程中发现impala并不兼容chdfs会上报不支持ofs。需要改动内核源码~。根据腾讯云同事提供的部分源码改动方案改完后，需要自己打包编译。下面大概记录下编译遇到的一些坑 E0312 14:35:09.345242 358875 impala-server.cc:285] Current
【impala】在容器里编译impala(4)
runzhliu大数据/容器日记
08-02 245
接【impala】在容器里编译impala(3)，也是最终篇了，反正我编译出来我需要的 impalad 了:) 主要是参考了 impala
impala的原理架构介绍及应用场景
01-27
简言之impala作为大数据实时查询分析工具，具有查询速度快，灵活性高，易整合，可伸缩性强等特点。1.查询速度快。Impala不同于hive，hive底层执行使用的是MapReduce引擎，仍然是一个批处理过程。不同于hive，impala...
python 安装impala包步骤
01-20
直接pip install impala 是不行滴，按照以下步骤安装就会成功！ 一路安装就可以 1、pip install six 2、pip install bit_array 3、pip install thriftpy 4、pip install thrift_sasl 5、pip install impyla 测试： ...
基于Impala的高性能数仓实践之执行引擎模块
大数据星球-浪尖
07-05 736
导读：本系列文章将结合实际开发和使用经验，聊聊可以从哪些方面对数仓查询引擎进行优化。Impala是Cloudera开发和开源的数仓查询引擎，以性能优秀著称。除了Apache Impala开源项目，业界知名的Apache Doris和StarRocks、SelectDB项目也跟Impala有千丝万缕的联系。笔者所在的网易数帆大数据团队，是最早一批将其作为分析型数仓查询引擎的...
Impala-shell相关源码笔记
Stiga Huang的专栏
04-28 591
Impala-shell是用python开发的，python代码不用声明变量类型，在看代码时很容易因为找不到定义而迷失。IDE能帮不少忙，但打开impala源码总还有些地方无法跳转。impala的开发者大多用vim、emacs这些工具，导致整个源码结构对IDE不太友好。另外有些源码是thrift生成的，需要编译完才有。总总这些，导致每次要改impala-shell代码时都要先花点时间回顾代码，这次决定写点笔记，帮助快速回忆/入门 impala-shell 源码。 Impala中的Python代码 Impal
Impala编译部署-4
a6418720的博客
05-16 512
这一步的目的是将程序运行必须的文件拷贝出来，便于发布和部署使用，我只想测试kudu，这里只拷贝了kudu的库。 /etc/default　　:准备放impala配置文件 /etc/hadoop　　：hadoop配置文件 be　　　　　　：impala运行文件 lib64　　　　：impala运行依赖库 hadoop　　　　：hadoop运行文件 dependency　　：ja...
查看详细_0587-6.1.0-CM 管理界面中Impala 的查看SQL查询详细异常问题分析
weixin_39720510的博客
01-13 625
作者：唐辉1.文档编写目的本篇文章主要介绍在Cloudera Manager 界面中Impala 查询详细界面显示无法检索此查询的详细信息问题的原因和解决办法内容概述1.文档说明2.问题描述3.问题分析4.解决办法测试环境1.CM和CDH版本为CDH 6.1.02.操作系统版本为RedHat7.22.问题描述在Cloudera Manager (以下简称CM)的管理界面有提供快速查看到Impala...
Impala查询报错：InternalException: Memory limit exceeded: Error occurred on backend
小海的专栏
10-28 1261
在HUE执行Impala查询的时候报错：InternalException: Memory limit exceeded: Error occurred on backend.... 报错原因是impala连接数过多导致的，可以通过修改连接数，或者等其他连接释放
impala 遇到的坑
weixin_33852020的博客
11-19 1517
文章原文链接 http://www.bbgo.xyz/accounts/... 1、尽量少使用 invalidate metadata，尽量用REFRESH TABLE_NAME;2、set APPX_COUNT_DISTINCT=true 与 ndv 函数是一样的，都只是估值 Impala SQL 不支持的一个查询中的多个聚合函数使用 ...
在CDH6.3中单独升级Impala到Apache Impala 3.4
Stiga Huang的专栏
06-20 5307
CDH6.3自带的Impala版本较低，是3.2版本加上一些Cloudera自己打的补丁。本文简述如何在CDH6.3中将Impala升级到Apache Impala 3.4。
impala insert overwrite并发冲突怎么解决
04-24
对于 Impala 的 insert overwrite 并发冲突问题，可以通过以下方法进行解决： 1. 在 Impala 中使用 ACID（原子性、一致性、隔离性和持久性）的特性，可以使用 Impala 的事务来保证数据的一致性。 2. 在 Impala 中可以使用 HDFS 的写前日志（Write Ahead Log，WAL）来记录写操作，从而避免并发冲突。 3. 在 Impala 中可以使用分布式锁来保证写操作的排他性，从而避免并发冲突。 4. 在 Impala 中可以使用多版本并发控制（MVCC）来处理并发的写操作，从而避免并发冲突。 以上方法都可以用来解决 Impala 的 insert overwrite 并发冲突问题，具体选择哪种方法需要考虑实际情况和需求。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值