每天一条Linux命令(22) wget (命令行下载工具)

---

在Linux系统中，命令wget用于从网络上下载某些资料，该命令对于能够连接到互联网的Linux系统的作用非常大，可以直接从网络上下载自己所需要的文件。

---

wget命令特点：

• 支持断点下载功能

• 支持FTP和HTTP下载方式。

• 支持代理服务器。非常稳定，它在带宽很窄的情况下或不稳定的网络中有很强的适应性。如果是由于网络的原因下载失败，wget会不断地尝试，直到整个文件下载完毕。如果是服务器打断了下载过程，它会再次连接到服务器上从停止的地方继续下载。这对那些从限定了连接时间的服务器上下载大文件非常有用。

---

语法：

wget [选项] [URL]

---

参数说明：

-o  将命令的执行结果写入文件

-O  指定保存的文件名后下载文件

--limit-rate  限速下载

-b  转入后台执行命令

-c  断点续传

--user-agent  指定客户端标志

-q  关闭下载时的输出

--tries=number  设置重试次数

--spider  模拟爬虫访问

---

案例：

使用wget 下载mysql

[root@master test04]# ls[root@master test04]# wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm--2020-08-12 14:02:43--  http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpmResolving repo.mysql.com (repo.mysql.com)... 2.21.16.253Connecting to repo.mysql.com (repo.mysql.com)|2.21.16.253|:80... connected.HTTP request sent, awaiting response... 200 OKLength: 6140 (6.0K) [application/x-redhat-package-manager]Saving to: ‘mysql-community-release-el7-5.noarch.rpm’
100%[=================================================================================>] 6,140       3.36KB/s   in 1.8s
2020-08-12 14:02:46 (3.36 KB/s) - ‘mysql-community-release-el7-5.noarch.rpm’ saved [6140/6140]
[root@master test04]# lsmysql-community-release-el7-5.noarch.rpm[root@master test04]#

在下载的过程中会显示进度条，包含（下载完成百分比，已经下载的字节，当前下载的速度，剩余下载的时间）。

---

使用-O选项指定下载文件的保存文件名 这里还是下载mysql  命名为mysql.rpm​​​​​​​

[root@master test04]# lsmysql-community-release-el7-5.noarch.rpm[root@master test04]# wget -O ./mysql.rpm http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm--2020-08-12 14:05:12--  http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpmResolving repo.mysql.com (repo.mysql.com)... 2.21.16.253Connecting to repo.mysql.com (repo.mysql.com)|2.21.16.253|:80... connected.HTTP request sent, awaiting response... 200 OKLength: 6140 (6.0K) [application/x-redhat-package-manager]Saving to: ‘./mysql.rpm’
100%[=================================================================================>] 6,140       --.-K/s   in 0s
2020-08-12 14:05:13 (167 MB/s) - ‘./mysql.rpm’ saved [6140/6140]
[root@master test04]# lsmysql-community-release-el7-5.noarch.rpm  mysql.rpm[root@master test04]# 

wget默认会以最后一个符合“/”的后面的字符来命名，对于动态链接的下载文件名通常会不正确。为了解决这个问题，我们可以使用参数-O来指定一个文件名。

---

通过--limit-rate限速下载。这里限制为0.5k 也就是5B​​​​​​​

[root@master test04]# wget --limit-rate=0.5k https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz--2020-08-12 14:29:22--  https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xzResolving www.python.org (www.python.org)... 151.101.76.223, 2a04:4e42:12::223Connecting to www.python.org (www.python.org)|151.101.76.223|:443... connected.HTTP request sent, awaiting response... 200 OKLength: 17212420 (16M) [application/octet-stream]Saving to: ‘Python-3.6.8.tar.xz.1’0% [                             ] 8094   512B/s   eta 9h 25m

---

使用-c参数断点续传​​​​​​​

[root@master test04]# wget https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz--2020-08-12 14:14:02--  https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xzResolving www.python.org (www.python.org)... 151.101.76.223, 2a04:4e42:12::223Connecting to www.python.org (www.python.org)|151.101.76.223|:443... connected.HTTP request sent, awaiting response... 200 OKLength: 17212420 (16M) [application/octet-stream]Saving to: ‘Python-3.6.8.tar.xz.1’
 1% [>          #  这里强制断开下载，下面使用-c参数从断点处继续下载                                                                    ] 230,364     4.14KB/s  eta 1h 41m ^C[root@master test04]# wget -c https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz--2020-08-12 14:15:48--  https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xzResolving www.python.org (www.python.org)... 151.101.76.223, 2a04:4e42:12::223Connecting to www.python.org (www.python.org)|151.101.76.223|:443... connected.HTTP request sent, awaiting response... 206 Partial ContentLength: 17212420 (16M), 16999185 (16M) remaining [application/octet-stream]Saving to: ‘Python-3.6.8.tar.xz’1% [+>          # 这里出现+号，表示从上次的断点继续下载

---

使用wget-b后台下载文件

[root@master test04]# ls

[root@master test04]# wget -b http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpmContinuing in background, pid 111183.   # 表示已在后台进行下载Output will be written to ‘wget-log’.[root@master test04]# tail wget-log  # 通过查看日志方式验证是否已下载Resolving repo.mysql.com (repo.mysql.com)... 23.40.10.87Connecting to repo.mysql.com (repo.mysql.com)|23.40.10.87|:80... connected.HTTP request sent, awaiting response... 200 OKLength: 6140 (6.0K) [application/x-redhat-package-manager]Saving to: ‘mysql-community-release-el7-5.noarch.rpm’
     0K .....                                                 100% 28.6K=0.2s
2020-08-12 14:19:41 (28.6 KB/s) - ‘mysql-community-release-el7-5.noarch.rpm’ saved [6140/6140]
[root@master test04]# lsmysql-community-release-el7-5.noarch.rpm  wget-log[root@master test04]#

---

伪装代理名称下载​​​​​​​

[root@master test04]# ls[root@master test04]# wget --user-agent="Mozilla/5.0(Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrom/10.0.648.204 Safri/534.16" http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm--2020-08-12 14:25:10--  http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpmResolving repo.mysql.com (repo.mysql.com)... 23.223.230.120Connecting to repo.mysql.com (repo.mysql.com)|23.223.230.120|:80... connected.HTTP request sent, awaiting response... 200 OKLength: 6140 (6.0K) [application/x-redhat-package-manager]Saving to: ‘mysql-community-release-el7-5.noarch.rpm’
100%[=================================================================================>] 6,140       2.70KB/s   in 2.2s
2020-08-12 14:25:14 (2.70 KB/s) - ‘mysql-community-release-el7-5.noarch.rpm’ saved [6140/6140]
[root@master test04]#

有些网站会根据判断代理名称不是浏览器而拒绝你的下载请求，不过你可以通过--user-agent参数进行伪装。

---

监控网站URL是否正常的案例​​​​​​​

[root@master test04]# wget -q -T 3 --tries=1 --spider www.baidu.com[root@master test04]# echo $?0  # 返回0表示正常[root@master test04]#