单引号、双引号、反引号
(1)单引号’ '内剥夺所有字符的特殊含义,所有字符都是单纯的字符串而没有特殊功能。
如,$取参数等命令是无效的。
(2)双引号" "中除了字符串,特殊字符是没有被转义的。
如,$等特殊字符一样可以使用其功能。
(3)反引号``是命令替换,即把命令输出结果传给入变量。
[filter@idxdb1 ~/yy]$ cat test_yh.sh
#!/bin/bash
a="hello://123"
echo "$a"
echo '$a'
echo `ls -l`
[filter@idxdb1 ~/yy]$ ls
analysis_diary test test_yh.sh
[filter@idxdb1 ~/yy]$ sh -x test_yh.sh
hello://123
$a
total 12 drwxrwxr-x 2 filter filter 4096 May 10 14:26 analysis_diary drwxrwxr-x 4 filter filter 4096 May 29 16:41 test -rw-rw-r-- 1 filter filter 66 Jun 3 14:59 test_yh.sh
grep全局查找
Global search Regular Expression and Print out the line
#文件准备
[engine@client1v ~/yy/grep_test]$ ls
ceng1 file0.txt
[engine@client1v ~/yy/grep_test]$ pwd
/home/engine/yy/grep_test
[engine@client1v ~/yy/grep_test]$ cat file0.txt
hello world
vim is a idea using on servers
shell is a script language used by thousands of people
I think people like the two beautiful tools most of the time
余姚
[engine@client1v ~/yy/grep_test]$ ls ceng1/
file1.txt
[engine@client1v ~/yy/grep_test]$ cat ceng1/file1.txt
hello test grep
today is fine
a beautiful bird fly in the sky
#中文查找
#-r 查找当前目录和递归查找当前目录的子目录
[engine@client1v ~/yy/grep_test]$ grep '余姚' -r /home/engine/yy/grep_test/*
/home/engine/yy/grep_test/file0.txt:余姚
[engine@client1v ~/yy/grep_test]$ grep '余姚' -r /home/engine/yy/grep_test/file0.txt
余姚
[engine@client1v ~/yy/grep_test]$ grep '余姚' /home/engine/yy/grep_test/*
/home/engine/yy/grep_test/file0.txt:余姚
[engine@client1v ~/yy/grep_test]$ grep '余姚' /home/engine/yy/grep_test/file0.txt
余姚
[engine@client1v ~/yy/grep_test]$ grep '余姚' /home/engine/yy/grep_test/ceng1/*
[engine@client1v ~/yy/grep_test]$ grep '余姚' /home/engine/yy/grep_test/ceng1/file1.txt
#英文查找
[engine@client1v ~/yy/grep_test]$ grep beautiful /home/engine/yy/grep_test/*
/home/engine/yy/grep_test/file0.txt:I think people like the two beautiful tools most of the time
[engine@client1v ~/yy/grep_test]$ grep beautiful -r /home/engine/yy/grep_test/*
/home/engine/yy/grep_test/ceng1/file1.txt:a beautiful bird fly in the sky
/home/engine/yy/grep_test/file0.txt:I think people like the two beautiful tools most of the time
[engine@client1v ~/yy/grep_test]$ pwd
/home/engine/yy/grep_test
[engine@client1v ~/yy/grep_test]$ grep beautiful ./*
./file0.txt:I think people like the two beautiful tools most of the time
------------------------------------------------------------------------------------[以下是错误查找方式]
[engine@client1v ~/yy/grep_test]$ grep beautiful /home/engine/yy/grep_test
[engine@client1v ~/yy/grep_test]$ grep beautiful /home/engine/yy/grep_test/
[engine@client1v ~/yy/grep_test]$ grep beautiful .
[engine@client1v ~/yy/grep_test]$ grep beautiful ./
[engine@client1v ~/yy/grep_test]$
#-c 只输出匹配行的计数
[engine@client1v ~/yy/grep_test]$ grep beautiful -c /home/engine/yy/grep_test/*
/home/engine/yy/grep_test/ceng1:0
/home/engine/yy/grep_test/file0.txt:1
#-l 只列出匹配的文件名
[engine@client1v ~/yy/grep_test]$ grep beautiful -r -l /home/engine/yy/grep_test/*
/home/engine/yy/grep_test/ceng1/file1.txt
/home/engine/yy/grep_test/file0.txt
----------------------------------------------------
#-L 只列出不匹配的文件名
[engine@client1v ~/yy/grep_test]$ grep '余姚' -r -L /home/engine/yy/grep_test/*
/home/engine/yy/grep_test/ceng1/file1.txt
[engine@client1v ~/yy/grep_test]$ grep '余姚' -r -l /home/engine/yy/grep_test/*
/home/engine/yy/grep_test/file0.txt
#-w 只匹配整个单词,而不是字符串的一部分
[engine@client1v ~/yy1/grep_test]$ grep a -w /home/engine/yy/grep_test/ceng1/file1.txt
a beautiful bird fly in the sky
##today is fine这一行没有匹配
#grep pattern1 files | grep pattern2 显示既匹配 pattern1 又匹配pattern2 的行 [交集]
[engine@client1v ~/yy1/grep_test]$ grep a -w /home/engine/yy/grep_test/ceng1/file1.txt
a beautiful bird fly in the sky
[engine@client1v ~/yy1/grep_test]$ grep is -w /home/engine/yy/grep_test/ceng1/file1.txt
today is fine
[engine@client1v ~/yy1/grep_test]$ grep a -w /home/engine/yy/grep_test/ceng1/file1.txt | grep is -w
[engine@client1v ~/yy1/grep_test]$
seq生成数组
[engine@client1v ~test]$ seq 1 3 10
1
4
7
10
for i in {1..5}
do
echo $i;
done
for i in $(seq 1 5)
do
echo $i;
done
在某个目录及其子目录下查找某个文件名
#目录递归查找精确文件名
find . -name "hadoop.py"
#目录递归查找匹配文件名
find . -name "*util*"
#指定查找深度为1
find . -maxdepth 1 -name "*util*"
输出文件指定行
#准备文件
[engine@client1v ~/yy1/grep_test]$ cat test_line.txt
hello
this is a file
show you something
the thing may be related to shell
a useful tool called sed tail head
you could use them to find content in pointed lines
wait a minute
we will start as soon as possible
#输出文件第一行
[engine@client1v ~/yy1/grep_test]$ sed -n 1p test_line.txt
hello
#输出文件第一至第三行
[engine@client1v ~/yy1/grep_test]$ sed -n '1,3p' test_line.txt
hello
this is a file
show you something
#输出文件第一,第三,第四行
[engine@client1v ~/yy1/grep_test]$ sed -n '1p;3,4p' test_line.txt
hello
show you something
the thing may be related to shell
#输出文件开头两行[默认10行]
[engine@client1v ~/yy1/grep_test]$ head -2 test_line.txt
hello
this is a file
#输出文件末尾一行[默认10行]
[engine@client1v ~/yy1/grep_test]$ tail -1 test_line.txt
we will start as soon as possible
#实时监控文件内容增加,默认10行
tail -f test_line.txt
后台运行进程
nohup允许用户在退出登录以后仍可以继续执行命令
&指定程序在后台执行,即在执行的命令后加上&;作用和bg命令相同
fg把后台任务放前台执行
[engine@client221v ~/yy1/get_page_info]$ nohup ./post_crawler.sh &
[1] 7658
[engine@client221v ~/yy1/get_page_info]$ nohup: ignoring input and appending output to `nohup.out'
[engine@client221v ~/yy1/get_page_info]$ jobs -l
[1]+ 7658 Running nohup ./post_crawler.sh &
[engine@client221v ~/yy1/get_page_info]$ ps -aux | grep post_crawler.sh
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
engine 7658 0.0 0.0 107608 1500 pts/1 S 19:15 0:00 /bin/sh ./post_crawler.sh
engine 24721 0.0 0.0 103244 940 pts/1 S+ 19:35 0:00 grep post_crawler.sh
[engine@client221v ~/yy1/get_page_info]$ kill -9 7658
[1]+ Killed nohup ./post_crawler.sh
[engine@client221v ~/yy1/get_page_info]$ jobs -l
命令行处理中文编码
[sch@crawl_service]$ export LC_ALL='zh_CN.utf8'
查看磁盘剩余空间及当前目录所在磁盘&文件大小
sch@ds201 [14:56:09]:~/yy1/train_data
$ df -h
文件系统 容量 已用 可用 已用% 挂载点
/dev/mapper/VolGroup00-LogVol03 100G 90G 11G 90% /
devtmpfs 126G 0 126G 0% /dev
tmpfs 126G 0 126G 0% /dev/shm
tmpfs 126G 4.0G 122G 4% /run
tmpfs 126G 0 126G 0% /sys/fs/cgroup
/dev/sda1 1014M 156M 859M 16% /boot
/dev/sdd2 1.2T 1.2T 22G 99% /da2/sch/bigdata
/dev/mapper/VolGroup00-LogVol01 2.0G 33M 2.0G 2% /tmp
/dev/mapper/VolGroup00-LogVol02 10G 586M 9.5G 6% /var
/dev/md127 1.5T 1.4T 69G 96% /da2
tmpfs 26G 32K 26G 1% /run/user/1017
想查看当前目录train_data所在盘,找到当前目录或其上级目录有软连的目录,然后ll
sch@ds201 [14:58:39]:~/yy1/train_data
$ ll
lrwxrwxrwx 1 root root 11 8月 18 2017 sch -> /da2/sch
sch@tt4 [19:49:14]:/home/sch
$ df -h /home/sch
文件系统 容量 已用 可用 已用% 挂载点
/dev/sdb 37T 177G 37T 1% /da1
sch@ds201 [14:59:09]:~/yy1/train_data
$du -ha pre_vectors.bin
3.5G pre_vectors.bin