记录一下保存list数据为csv文件的方法。
数据格式
data = [ [url,title,click_number,html_content,crawl_time], … ]
http://finance.eastmoney.com/yaowen.html,要闻,,,2019-07-28 12:54:30
http://finance.eastmoney.com/a/ccjdd.html,财经导读,,,2019-07-28 12:54:30
http://finance.eastmoney.com/a/ccjxw.html,产经资讯,,,2019-07-28 12:54:31
http://finance.eastmoney.com/yaowen_cgnjj.html,国内经济,,,2019-07-28 12:54:31
http://finance.eastmoney.com/yaowen_cgjjj.html,国际经济,,,2019-07-28 12:54:31
http://finance.eastmoney.com/a/czqyw.html,证券要闻,,,2019-07-28 12:54:31
http://finance.eastmoney.com/company.html,上市公司,,,2019-07-28 12:54:31
http://finance.eastmoney.com/a/czsdc.html,纵深调查,,,2019-07-28 12:54:31
http://finance.eastmoney.com/a/crdsm.html,热点扫描,,,2019-07-28 12:54:31
http://finance.eastmoney.com/pinglun_cjjsp.html,经济时评,,,2019-07-28 12:54:32
http://finance.eastmoney.com/pinglun_ccyts.html,产业透视,,,2019-07-28 12:54:32
http://finance.eastmoney.com/pinglun_csygc.html,商业观察,,,2019-07-28 12:54:32
http://finance.eastmoney.com/pinglun_cgspl.html,股市评论,,,2019-07-28 12:54:32
http://finance.eastmoney.com/a/201907281190069120.html,人民日报钟声:故技重演不得人心,145927,,2019-07-28 12:55:03
http://finance.eastmoney.com/a/201907281190068761.html,二季度小微贷款余额同比增22.5%,4013,,2019-07-28 12:55:04
http://finance.eastmoney.com/a/201907271190034954.html,805份年度财报大抽查证监会揪“猫腻”,37365,,2019-07-28 12:55:05
http://finance.eastmoney.com/a/201907281190069235.html,29家企业IPO遭中止(附名单),204087,,2019-07-28 12:55:06
环境
- Ubuntu16.04
- pandas
pip install pandas
方法
import pandas as pd
data = [ ['http://finance.eastmoney.com/yaowen.html', '要闻', '', '', '2019-07-28 12:54:30'],
... ]
#保存数据
columns = ['url', 'title', 'click_number', 'html_content', 'crawl_time']
df = pd.DataFrame(columns = columns, data = data)
df.to_csv('data.csv', encoding='utf-8')
print('data_len:', len(data))
结果
查看前10行: head -10 ‘data.csv’
,url,title,click_number,html_content,crawl_time
0,http://finance.eastmoney.com/yaowen.html,要闻,,,2019-07-28 12:54:30
1,http://finance.eastmoney.com/a/ccjdd.html,财经导读,,,2019-07-28 12:54:30
2,http://finance.eastmoney.com/a/ccjxw.html,产经资讯,,,2019-07-28 12:54:31
3,http://finance.eastmoney.com/yaowen_cgnjj.html,国内经济,,,2019-07-28 12:54:31
4,http://finance.eastmoney.com/yaowen_cgjjj.html,国际经济,,,2019-07-28 12:54:31
5,http://finance.eastmoney.com/a/czqyw.html,证券要闻,,,2019-07-28 12:54:31
6,http://finance.eastmoney.com/company.html,上市公司,,,2019-07-28 12:54:31
7,http://finance.eastmoney.com/a/czsdc.html,纵深调查,,,2019-07-28 12:54:31
8,http://finance.eastmoney.com/a/crdsm.html,热点扫描,,,2019-07-28 12:54:31
9,http://finance.eastmoney.com/pinglun_cjjsp.html,经济时评,,,2019-07-28 12:54:32