python 来自互联网数据

最新推荐文章于 2024-07-30 17:23:09 发布

zianren

最新推荐文章于 2024-07-30 17:23:09 发布

阅读量99

点赞数

分类专栏：基于python的机器学习文章标签： python html

本文链接：https://blog.csdn.net/u012678323/article/details/118514106

版权

基于python的机器学习专栏收录该内容

9 篇文章 0 订阅

订阅专栏

#P69  Web Page
#Under Py35
import urllib
from bs4 import BeautifulSoup
import numpy as np

url = urllib.request.urlopen("http://interthing.org/");
#url = urllib.request.urlopen("https://www.burnaby.ca/test-page.html");
html = url.read()
soup = BeautifulSoup(html, "html5lib")
table = soup.find("table")
headings = [th.get_text() for th in table.find("tr").find_all("th")]

datasets = []
for row in table.find_all("tr")[2:]:
    dataset = list(zip(headings, (td.get_text() for td in row.find_all("td"))))
    datasets.append(dataset)

nd=np.array(datasets)
features=nd[:,1:,1].astype('float')
targets=(nd[:,0,1:]).astype('str')
print(features)
print(targets)

Python 有个名为BeautifulSoup的优秀的，用于从HTML和XML文档中提取数据。

BeautifulSoup （美丽的汤）重要的包，用来解析网页，后来装的；pip list 可看版本

urllib 用来爬虫，网页非常重要的一个lib

html = url.read() 先来读去网页
soup = BeautifulSoup(html, "html5lib") 之后解析网页
table = soup.find("table") 发现网页表格

#p69需要改。网址；1变成2；

for row in table.find_all("tr")[2:]: 把1改成2 （？）

运行结果：

可以右键查看源文件：

<html>
<head>
	<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
	<title>Test Page With Lorem</title>
</head>
<body>
<div style="width: 80%">
<h3>Lorem</h3>

<table border="1" cellpadding="0" cellspacing="0" style="width:150px;float: right">
	<caption style="border:solid 1px black"><strong>DataTable</strong></caption>
	<tbody>
	<th>Species</th><th>data1</th><th>data2</th>	
    <tr>
<tr valign="top" class="Failure">
			<td>Thrip</td>
			<td>-.07</td>
			<td>.21</td>
		</tr>
		<tr>
			<td>Whitefly</td>
			<td>.78</td>
			<td>.36</td>
		</tr>
		<tr>
			<td>Aphid</td>
			<td>.64</td>
			<td>.29</td>
		</tr>
	</tbody>
</table>

<p style="margin: 0px 0px 15px; padding: 0px; border: 0px; outline: 0px; font-size: 14px; color: rgb(51, 51, 51); font-family: Geneva, Tahoma, 'Nimbus Sans L', sans-serif; line-height: 23.8px; background: rgb(255, 255, 255);"><span style="margin: 0px; padding: 0px; border: 0px; outline: 0px; font-family: Arial, Helvetica, sans; line-height: 14px; text-align: justify; background: transparent;">&quot;Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.&quot;</span></p>

<p style="margin: 0px 0px 15px; padding: 0px; border: 0px; outline: 0px; font-size: 14px; color: rgb(51, 51, 51); font-family: Geneva, Tahoma, 'Nimbus Sans L', sans-serif; line-height: 23.8px; background: rgb(255, 255, 255);"><span style="margin: 0px; padding: 0px; border: 0px; outline: 0px; font-family: Arial, Helvetica, sans; line-height: 14px; text-align: justify; background: transparent;"><span style="margin: 0px; padding: 0px; border: 0px; outline: 0px; background: transparent;">&quot;Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.&quot;</span><span style="margin: 0px; padding: 0px; border: 0px; outline: 0px; background: transparent;">&quot;Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.&quot;</span></span></p>
</div>
</body>
</html>

会找各种标签；HTML的各种标签，怎么编写，十几个；（补）

table标签 tr td

目前有反爬虫，会封IP。或直接价格使用图片显示。

sy07b补充代码为 #chap07 Python正则表达式及使用32

zianren

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
python 来自互联网数据

#P69 Web Page#Under Py35import urllibfrom bs4 import BeautifulSoupimport numpy as npurl = urllib.request.urlopen("http://interthing.org/");#url = urllib.request.urlopen("https://www.burnaby.ca/test-page.html");html = url.read()soup = Beautifu...
复制链接

扫一扫

专栏目录