GBase 8a 查询结果导出到hdfs报错问题排查

一、查询结果导出到hdfs需要特别注意的事项

查询结果导出语句示例：

create table t(a bigint,b varchar(20)) distributed by ('a');

insert into t values(10,'aaa'),(20,'bbb'),(30,'ccc'),(40,'ddd'),(50,'eee');

set gbase_hdfs_namenodes="ip1,ip2" ;

show variables like 'gbase_hdfs_namenodes';

select * from test.t into outfile 'hdp://hdfs@ip1:ip1_port/output/t.tbl' outfilemode by hdfs writemode by overwrites;

注意：如果导出时遇到报错：failed with error - Fail to Create HDFS File，先排查部署hadoop环境的节点Namenode和Datanode服务是否正常，若服务未启动则需要重新启动，如果服务显示正常，则需要排查集群各节点/etc/hosts中是否添加了Hadoop的Namenode和Datanode的IP地址和主机名映射，

例如：

[gbase@www-node-1 gcinstall]$ cat /etc/hosts

ip1 www-node-1

ip2 www-node-2

二、查询结果导出到hdfs曾遇到的问题

[hdfs@www-node-1 hadoop-3.1.0]$ jps

OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085c00000, 5767168, 0) failed; error='Cannot allocate memory' (errno=12)## There is insufficient memory for the Java Runtime Environment to continue.# Native memory allocation (mmap) failed to map 5767168 bytes for committing reserved memory.# An error report file with more information is saved as:# /home/hdfs/hadoop/hadoop-3.1.0/hs_err_pid21392.log

问题原因：jvm分配的内存不够

解决办法：修改配置文件参数

/home/hdfs/hadoop/hadoop-3.1.0/etc/hadoop/hadoop-env.sh

添加内容：

export HADOOP_HEAPSIZE_MAX=4G（根据具体情况进行设置即可）

然后，执行停启服务，使修改后的配置文件生效

cd ${HADOOP_HOME}

sbin/stop-dfs.sh

sbin/start-dfs.sh