16.处理CSV文件和JSON数据
CSV—Comma-Separated Values 逗号分隔的值。
JSON——JavaScript Object Notation JS对象
1)CSV模块
CSV对数据中的逗号要进行转义。
csv模块是python自带,不需要安装
a)reader对象用于读取
csv.reader(open(fname))
csv.reader().line_num 当前查询的行数
csvdata = list(csv.reader()) 数据可作为列表来处理
!!!reader对象只能循环遍历一次,如再次读取,必须再创建。
b)writer对象用于写入
测试程序:test_1601.py
import csv,os
os.chdir('d:/temp/waffle')
opFile = open('oput.csv','w',newline='')
opWriter = csv.writer(opFile,delimiter='!',lineterminator='\n\n')
opWriter.writerow(['spam','eggs','bacon','ham'])
opWriter.writerow(['hello,world!','eggs','bacon','ham'])
opWriter.writerow([1,2,3.141592,4])
opWriter.writerow([1,2,3.141592,4,3])
opWriter.writerow([1,2,3.141592,4,3,2])
opFile.close()
csv.writer(open(ofn,’w’,newline=’’)
writerow(listvalue)
ofn.close()
!!writer不强制写入行的列数一致。可以是不同列数的列表。
!!newline=’’ 是解决windows下的换行符与其它系统不一致的问题
!!数据内容中有逗号,则用双引号括住。
c)delimiter、lineterminator关键字参数
delimiter 用于修改分隔符,比如换成‘\t’
lineterminator用于修改行终止字符,比如替换成‘\n\n’
d)DictReader和DictwriterCSV对象
用于有标题行的CSV文件的读取和写入。
DictReader用于带标题行CSV文件的读取
行数据采用字典方式,取数据时,以标题名称为索引读取。
如果文件中没有标题行,在生成DictReader对象时,可以带入标题名称列表,作为第二个参数。
DictWriter用于写入带标题行的CSV文件
DictWriter.writeheader() 写入标题行
DictWriter.writerow() 写数据行
测试程序:test_1602.py
import csv,os
os.chdir('d:/temp/waffle')
outputFile = open('oputhead.csv','w',newline='')
outputDictWriter = csv.DictWriter(outputFile,['Name','Pet','Phone'])
outputDictWriter.writeheader()
outputDictWriter.writerow({'Name':'Alice','Pet':'cat','Phone':'555-1234'})
outputDictWriter.writerow({'Name':'Bob','Phone':'555-9999'})
outputDictWriter.writerow({'Phone':'555-5555','Name':'Carol','Pet':'dog'})
outputFile.close()
2)项目:从CSV文件中删除标题行
测试程序:test_1603.py
#! python3
# removeCsvHeader.py - Removes the header from all CSV files in the current working directory.
import csv,os
os.chdir('d:/temp/removeCsvHeader')
os.makedirs('headerRemoved',exist_ok=True)
# Loop throuth every file in the current working directory.
for csvFilename in os.listdir('.'):
if not csvFilename.endswith('.csv'):
continue # skip non-csv files
print('Removing header form ' + csvFilename + '...')
# Read the CSV file in (skipping first row).
csvRows = []
csvFileObj = open(csvFilename)
readerObj = csv.reader(csvFileObj)
for row in readerObj:
if readerObj.line_num == 1:
continue # skip first row
csvRows.append(row)
csvFileObj.close()
# write out the CSV file.
csvFileObj = open(os.path.join('headerRemoved',csvFilename),'w',newline='')
csvWriter = csv.writer(csvFileObj)
for row in csvRows:
csvWriter.writerow(row)
csvFileObj.close()
3)JSON和API
JSON格式:JavaScript对象表示法。
4)json模块
json.loads() 装入数据
json.dumps() 卸出数据
!!json字符串数据表示用双引号,不是单引号
5)项目:取得当前天气数据
URL示例:
https://api.openweathermap.org/data/2.5/forecast? q=San%20Francisco,US&cnt=3&APPID=XXXXX
要求输入城市名称和国家代码,国家代码采用ISO 3166(书中的CA意为加州,CA国家代码为加拿大,改成US)
需要先去网站注册一个免费用户,APPID上带入密钥才能访问
支持三种请求报文格式:
api.openweathermap.org/data/2.5/forecast?q={city name}&appid={API key}
api.openweathermap.org/data/2.5/forecast?q={city name},{country code}&appid={API key}
api.openweathermap.org/data/2.5/forecast?q={cityname},{statecode},{countrycode}&appid={API key}
测试程序:test_1604.py
#! python3
# getOpenWeather.py - Prints the weather for a location from the command line.
APPID = 'XXXXXXXXXXXX'
import json,requests,sys
# Compute location from command line arguments.
if len(sys.argv) < 2:
print('Usage: getOpenWeather.py city_name,2-letter_country_code')
sys.exit()
location = ' '.join (sys.argv[1:])
# Download the Json data from OpenWeatherMap.org's API.
url = 'https://api.openweathermap.org/data/2.5/forecast?q=%s&cnt=3&APPID=%s' % (location,APPID)
response = requests.get(url)
response.raise_for_status()
# Uncomment to see the raw JSON text:
#print(response.text)
# Load JSON data into a Python variable.
weatherData = json.loads(response.text)
# Print weather descriptions.
w = weatherData['list']
print('Current weather in %s:' % (location))
print(w[0]['weather'][0]['main'],'-',w[0]['weather'][0]['description'])
print()
print('Tomorrow:')
print(w[1]['weather'][0]['main'],'-',w[1]['weather'][0]['description'])
print()
print('Day after tomorrow:')
print(w[2]['weather'][0]['main'],'-',w[2]['weather'][0]['description'])