qiulp@qiuluping:~/hadoop_work/hadoop-1.0.3_qiu$ ./bin/hadoop fs -ls /
Exception in thread "Main Thread" java.lang.Error: Unresolved compilation problems:
The import org.apache.commons cannot be resolved
The import org.apache.commons cannot be resolved
The import org.codehaus cannot be resolved
The import org.codehaus cannot be resolved
Log cannot be resolved to a type
LogFactory cannot be resolved
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
Log cannot be resolved to a type
JsonFactory cannot be resolved to a type
JsonFactory cannot be resolved to a type
JsonGenerator cannot be resolved to a type
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:58)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:57)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:1895)
修改了主机目录后,出现问题,重新copy就好了。- hadoop streaming
日志级别问题
- 误用root启动hadoop时,
- 用root重启hadoop时,显示连接不上 查看日志..
- 帮波波装时候,删除localhost
- export HADOOP_HOME_WARN_SUPPRESS=1
- 网络访问不到某个节点 防火墙没有关闭 访问量很大时起作用
网络访问不到 :端口开了访问不到
或者端口没开
体现在MR就是程序会在某个地方卡很久
reduce > copy (5 of 6 at 0.00 MB/s)
- mjiang@syvenus:~/program/eclipse/customer$ hadoop --config ~/hadoop_work/hadoop-1.0.3/conf fs -ls /
Found 16 items
-rw-r--r-- 3 qiulp supergroup 446615 2012-08-24 17:18 /CHANGES.txt
drwxrwxrwx - hadoop supergroup 0 2012-09-19 17:30 /data1
drwxrwxrwx - hadoop supergroup 0 2012-08-30 19:25 /fenxi_system
drwxrwxrwx - hadoop supergroup 0 2012-04-09 17:50 /hadoop
drwxrwxrwx - hadoop supergroup 0 2012-09-19 13:28 /home
drwxrwxrwx - hadoop supergroup 0 2012-07-26 17:45 /liujingyu_t1
drwxrwxrwx - hadoop supergroup 0 2012-07-26 17:45 /liujingyu_t2
drwxrwxrwx - hadoop supergroup 0 2012-07-26 13:42 /liujingyu_test
drwxrwxrwx - mjiang supergroup 0 2012-09-10 18:51 /o3
drwxrwxrwx - hadoop supergroup 0 2012-09-11 10:06 /o5
drwxr-xr-x - qiulp supergroup 0 2012-08-28 16:48 /qiulp
drwxrwxrwx - hadoop supergroup 0 2012-09-05 13:40 /system
drwxr-xr-x - mjiang supergroup 0 2012-08-30 10:44 /t_fx_latitude_daily
drwxrwxrwx - hadoop supergroup 0 2012-02-14 10:51 /test
drwxrwxrwx - hadoop supergroup 0 2012-09-19 17:30 /tmp
drwxrwxrwx - hadoop supergroup 0 2012-09-19 17:28 /user
mjiang@syvenus:~/program/eclipse/customer$ hadoop fs -ls /
Found 26 items
dr-xr-xr-x - root root 0 2012-09-17 14:41 /proc
drwxr-xr-x - root root 4096 2012-09-17 14:13 /home
drwxr-xr-x - root root 4096 2012-04-19 17:32 /mnt
drwxr-xr-x - root root 0 2012-09-17 14:41 /sys
drwxr-xr-x - root root 800 2012-09-18 22:46 /run
drwxr-xr-x - root root 4096 2012-09-17 14:25 /usr
-rw-r--r-- 1 root root 14189274 2012-07-24 20:56 /initrd.img
drwxr-xr-x - root root 4096 2012-04-26 00:04 /lib64
drwx------ - root root 16384 2012-07-24 13:41 /lost+found
drwxr-xr-x - root root 4096 2012-09-14 19:32 /var
drwxr-xr-x - root root 4096 2012-07-24 13:43 /cdrom
-rw-r--r-- 1 root root 4965840 2012-04-26 08:13 /vmlinuz
drwxr-xr-x - root root 4096 2012-07-24 20:56 /lib
drwxr-xr-x - root root 4096 2012-04-26 00:04 /srv
drwxr-xr-x - root root 4096 2012-09-18 18:46 /media
drwxr-xr-x - root root 4096 2012-09-13 10:44 /sbin
drwxrwxrwx - root root 20480 2012-09-20 09:18 /tmp
drwxr-xr-x - root root 4096 2012-07-30 11:19 /opt
drwxr-xr-x - root root 4096 2012-07-24 20:56 /bin
drwx------ - root root 4096 2012-09-17 14:25 /root
drwxr-xr-x - root root 4096 2012-09-12 18:15 /123
drwxr-xr-x - root root 4280 2012-09-19 14:55 /dev
drwxr-xr-x - root root 12288 2012-09-19 20:16 /etc
drwxr-xr-x - root root 4096 2012-09-14 18:01 /test1
drwxr-xr-x - root root 1024 2012-07-24 20:56 /boot
drwxr-xr-x - root root 4096 2012-03-06 01:54 /selinux
mjiang@syvenus:~/hadoop_work/hadoop-1.0.3$ vi bin/hadoop-config.sh
mjiang@syvenus:~/hadoop_work/hadoop-1.0.3$ echo $HADOOP_CONF_DIR
/home/yibo/hadoop/hadoop-1.0.3/conf