上几天跑程序,hadoop集群报了如下错误:
2011-08-05 11:57:03,067 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:Started memstore flush forwb_userinfo,sina1733680271,1312516509841.54e4e40f620536d9a23a178746dee428.,current region memstore size 206.0m
2011-08-05 11:57:03,067 DEBUGorg.apache.hadoop.hbase.regionserver.HRegion: Finished snapshotting, commencingflushing stores
2011-08-05 11:57:59,836 FATALorg.apache.hadoop.hbase.regionserver.wal.HLog: Could not append. Requestingclose of hlog
2011-08-05 11:58:20,044 INFOorg.apache.hadoop.hbase.regionserver.wal.HLog: regionserver60020.logSyncerexiting
开始以为是程序问题,后来经过排查,发现是hadoop打开文件数过多导致,同时在新集群环境下没有设置ulimit数,
修改:
vi /etc/security/limits.conf
在文件最后添加如下:
hbase - nofile 32768
然后重启hadoop,问题解决!!