删除文件中重复的行

今天在经过多次执行脚本后时候突然发现：/etc/hosts下面有好多重复的行，突然想起来之前记得学过有命令的，但是忘了，后来在群里面吼了几句，有好多牛人相应，闲来与大家分享，O(∩_∩)O~

[root@zy zy]# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1               localhost.localdomain localhost puppet
192.168.55.229          master.puppet.com master
192.168.55.230          node1.puppet.com node1
192.168.55.231          node2.puppet.com node2
192.168.55.81           puppetmaster.99bill.com
192.168.55.229          zy.99bill.com  zy
192.168.55.229          zy.99bill.com  zy
192.168.55.229          zy.99bill.com  zy

# cat /etc/hosts | sort | uniq

[root@zy zy]# cat /etc/hosts | sort | uniq
127.0.0.1               localhost.localdomain localhost puppet
192.168.55.229          master.puppet.com master
192.168.55.229          zy.99bill.com  zy
192.168.55.230          node1.puppet.com node1
192.168.55.231          node2.puppet.com node2
192.168.55.81           puppetmaster.99bill.com
# Do not remove the following line, or various programs
# that require network functionality will fail.

注意查看：顺序发生了变化，这是因为sort的原因。

 

那么我们可以直接执行：

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1               localhost.localdomain localhost puppet
192.168.55.229          master.puppet.com master
192.168.55.230          node1.puppet.com node1
192.168.55.231          node2.puppet.com node2
192.168.55.81           puppetmaster.99bill.com
192.168.55.229          zy.99bill.com  zy

注意：这次我们发现只是重复的行给删除了，只保留了一行。

 

另外：

# cat /etc/hosts | sort -u

可以直接删除重复的,嘿嘿。。。