格式化(format)hadoop(暂时没发现的特别的作用):
wyq@wyq-Rev-1-0 ~/Documents/software/hadoop-0.20.203.0 $ bin/hadoop namenode -format
启动hadoop:
wyq@wyq-Rev-1-0 ~/Documents/software/hadoop-0.20.203.0 $ bin/start-all.sh
jsp查看java相关的进程
将用户下的内容拷贝到hdfs中,是在/user/wyq目录下的。
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -put /home/wyq/Documents/example/input in
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -put /home/wyq/Documents/example/input user
快速找到in文件夹
wyq@wyq-Rev-1-0 ~/Documents/example/input $ hadoop dfs -ls ./in/*\
> test1.txt
Found 1 items
-rw-r--r-- 1 wyq supergroup 12 2014-10-09 20:57 /user/wyq/in/test1.txt
需要先启动hadoop,然后利用hadoop的jar包中的wordcount示例跑程序:
wyq@wyq-Rev-1-0 ~/Documents/software/hadoop-0.20.203.0 $ bin/hadoop jar hadoop-examples-0.20.203.0.jar wordcount in out
查看hadoop上有哪些目录 hadoop fs -ls /
直接打开较深层次的目录:
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -ls
Found 3 items
drwxr-xr-x - wyq supergroup 0 2014-10-09 20:57 /user/wyq/in
drwxr-xr-x - wyq supergroup 0 2014-10-13 11:06 /user/wyq/out
drwxr-xr-x - wyq supergroup 0 2014-10-09 20:58 /user/wyq/user
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -ls ./out
查看目录中的文件内容:
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -cat ./out/*
hadoop 1
hello 2
world 1
cat: File does not exist: /user/wyq/out/_logs
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -cat part-r-00000
cat: File does not exist: /user/wyq/part-r-00000
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -cat /out/part-r-00000
cat: File does not exist: /out/part-r-00000
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -cat ./out/part-r-00000
hadoop 1
hello 2
world 1
hadoop中没有当前目录的概念,即不能用cd来切换目录。只能用ls将路径一直查下去。
名称节点(namenode)起着控制整个hdfs的作用。jobtracker显示hadoop的工作情况。
jobtracker.jsp: http://localhost:50030
hdfs的监控: http://localhost:50070
hadoop中的文件内容存储在实际系统中的文件样子:
wyq@wyq-Rev-1-0 ~/Documents/software/hadoop-0.20.203.0/data $ ls -lR
.:
total 16
drwxrwxr-x 2 wyq wyq 4096 Feb 21 2014 current
drwxrwxr-x 2 wyq wyq 4096 Feb 21 2014 detach
-rw-rw-r-- 1 wyq wyq 157 Feb 21 2014 storage
drwxrwxr-x 2 wyq wyq 4096 Feb 21 2014 tmp
./current:
total 4
-rw-rw-r-- 1 wyq wyq 0 Feb 21 2014 dncp_block_verification.log.curr
-rw-rw-r-- 1 wyq wyq 155 Feb 21 2014 VERSION
./detach:
total 0
./tmp:
total 0
将hdfs中文件拷贝到用户系统下:
hdfs dfs -get /data/test/monitorData(或者直接写该用户下的文件名) ./
删除hdfs中的文件
hdfs dfs -rmr /data/test/monitorData
查看HDFS基本统计信息:
hadoop dfsadmin -report
进入和退出安全模式:(hdfs中的块是否达到了安全的比例)
hadoop dfsadmin -safemode enter
hadoop dfsadmin -safemode leave
负载均衡:start-balancer.sh
HDFS API:
http://hadoop.apache.org/hdfs/docs/r0.22.0/api/index.html
Hadoop命令大全:
hadoop.apache.org/common/docs/stable/commands_manual.html
hadoop.apache.org/common/docs/stable/api/index.html
hdfs参数的绝对路径:
hdfs://localhost:9000/user/hadoop/input hdfs://localhost:9000/user/hadoop/output
在hadoop上创建一个目录 hadoop fs -mkdir /user hadoop fs -mkdir /user/hadoop
查看hadoop中的特定目录 hadoop fs -ls /user hadoop fs -ls /user/hadoop
运行jar包 hadoop jar hdfs1.jar com.itcast.hadoop.hdfs.HDFSMkdir hadoop jar hadoop-examples-0.20.203.0.jar wordcount /tmp/input /tmp/output
//需要看hadoop-examples-0.20.203.0.jar与jar包是否在同一个目录下
查看txt hadoop fs -text /user/hadoop/20140113/word.txt
相当于cat功能 hadoop fs -cat /output/
-put相当于复制的功能 hadoop fs -put input/* /tmp/input
echo与管道的结合 echo ”hello hadoop bye hadoop” >> ~/input/file2
http://localhost:50030/jobtracker.jsp
http://localhost:50070/
wyq@wyq-Rev-1-0 ~/Documents/software/hadoop-0.20.203.0 $ bin/hadoop namenode -format
启动hadoop:
wyq@wyq-Rev-1-0 ~/Documents/software/hadoop-0.20.203.0 $ bin/start-all.sh
jsp查看java相关的进程
将用户下的内容拷贝到hdfs中,是在/user/wyq目录下的。
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -put /home/wyq/Documents/example/input in
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -put /home/wyq/Documents/example/input user
快速找到in文件夹
wyq@wyq-Rev-1-0 ~/Documents/example/input $ hadoop dfs -ls ./in/*\
> test1.txt
Found 1 items
-rw-r--r-- 1 wyq supergroup 12 2014-10-09 20:57 /user/wyq/in/test1.txt
需要先启动hadoop,然后利用hadoop的jar包中的wordcount示例跑程序:
wyq@wyq-Rev-1-0 ~/Documents/software/hadoop-0.20.203.0 $ bin/hadoop jar hadoop-examples-0.20.203.0.jar wordcount in out
查看hadoop上有哪些目录 hadoop fs -ls /
直接打开较深层次的目录:
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -ls
Found 3 items
drwxr-xr-x - wyq supergroup 0 2014-10-09 20:57 /user/wyq/in
drwxr-xr-x - wyq supergroup 0 2014-10-13 11:06 /user/wyq/out
drwxr-xr-x - wyq supergroup 0 2014-10-09 20:58 /user/wyq/user
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -ls ./out
查看目录中的文件内容:
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -cat ./out/*
hadoop 1
hello 2
world 1
cat: File does not exist: /user/wyq/out/_logs
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -cat part-r-00000
cat: File does not exist: /user/wyq/part-r-00000
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -cat /out/part-r-00000
cat: File does not exist: /out/part-r-00000
wyq@wyq-Rev-1-0 ~ $ hadoop dfs -cat ./out/part-r-00000
hadoop 1
hello 2
world 1
hadoop中没有当前目录的概念,即不能用cd来切换目录。只能用ls将路径一直查下去。
名称节点(namenode)起着控制整个hdfs的作用。jobtracker显示hadoop的工作情况。
jobtracker.jsp: http://localhost:50030
hdfs的监控: http://localhost:50070
hadoop中的文件内容存储在实际系统中的文件样子:
wyq@wyq-Rev-1-0 ~/Documents/software/hadoop-0.20.203.0/data $ ls -lR
.:
total 16
drwxrwxr-x 2 wyq wyq 4096 Feb 21 2014 current
drwxrwxr-x 2 wyq wyq 4096 Feb 21 2014 detach
-rw-rw-r-- 1 wyq wyq 157 Feb 21 2014 storage
drwxrwxr-x 2 wyq wyq 4096 Feb 21 2014 tmp
./current:
total 4
-rw-rw-r-- 1 wyq wyq 0 Feb 21 2014 dncp_block_verification.log.curr
-rw-rw-r-- 1 wyq wyq 155 Feb 21 2014 VERSION
./detach:
total 0
./tmp:
total 0
将hdfs中文件拷贝到用户系统下:
hdfs dfs -get /data/test/monitorData(或者直接写该用户下的文件名) ./
删除hdfs中的文件
hdfs dfs -rmr /data/test/monitorData
查看HDFS基本统计信息:
hadoop dfsadmin -report
进入和退出安全模式:(hdfs中的块是否达到了安全的比例)
hadoop dfsadmin -safemode enter
hadoop dfsadmin -safemode leave
负载均衡:start-balancer.sh
HDFS API:
http://hadoop.apache.org/hdfs/docs/r0.22.0/api/index.html
Hadoop命令大全:
hadoop.apache.org/common/docs/stable/commands_manual.html
hadoop.apache.org/common/docs/stable/api/index.html
hdfs参数的绝对路径:
hdfs://localhost:9000/user/hadoop/input hdfs://localhost:9000/user/hadoop/output
在hadoop上创建一个目录 hadoop fs -mkdir /user hadoop fs -mkdir /user/hadoop
查看hadoop中的特定目录 hadoop fs -ls /user hadoop fs -ls /user/hadoop
运行jar包 hadoop jar hdfs1.jar com.itcast.hadoop.hdfs.HDFSMkdir hadoop jar hadoop-examples-0.20.203.0.jar wordcount /tmp/input /tmp/output
//需要看hadoop-examples-0.20.203.0.jar与jar包是否在同一个目录下
查看txt hadoop fs -text /user/hadoop/20140113/word.txt
相当于cat功能 hadoop fs -cat /output/
-put相当于复制的功能 hadoop fs -put input/* /tmp/input
echo与管道的结合 echo ”hello hadoop bye hadoop” >> ~/input/file2
http://localhost:50030/jobtracker.jsp
http://localhost:50070/