【DS实践 | Coursera】Assignment 2 | Applied Plotting, Charting & Data Representation in Python

---

一、问题分析

1.1 问题描述

Before working on this assignment please read these instructions fully. In the submission area, you will notice that you can click the link to Preview the Grading for each step of the assignment. This is the criteria that will be used for peer grading. Please familiarize yourself with the criteria before beginning the assignment.

An NOAA dataset has been stored in the file data/C2A2_data/BinnedCsvs_d400/fb441e62df2d58994928907a91895ec62c2c42e6cd075c2700843b89.csv. This is the dataset to use for this assignment. Note: The data for this assignment comes from a subset of The National Centers for Environmental Information (NCEI) Daily Global Historical Climatology Network (GHCN-Daily). The GHCN-Daily is comprised of daily climate records from thousands of land surface stations across the globe.

Each row in the assignment datafile corresponds to a single observation.

The following variables are provided to you:

• id : station identification code
• date : date in YYYY-MM-DD format (e.g. 2012-01-24 = January 24, 2012)
• element : indicator of element type
  • TMAX : Maximum temperature (tenths of degrees C)
  • TMIN : Minimum temperature (tenths of degrees C)
• value : data value for element (tenths of degrees C)

For this assignment, you must:

1. Read the documentation and familiarize yourself with the dataset, then write some python code which returns a line graph of the record high and record low temperatures by day of the year over the period 2005-2014. The area between the record high and record low temperatures for each day should be shaded.
2. Overlay a scatter of the 2015 data for any points (highs and lows) for which the ten year record (2005-2014) record high or record low was broken in 2015.
3. Watch out for leap days (i.e. February 29th), it is reasonable to remove these points from the dataset for the purpose of this visualization.
4. Make the visual nice! Leverage principles from the first module in this course when developing your solution. Consider issues such as legends, labels, and chart junk.

The data you have been given is near Ann Arbor, Michigan, United States, and the stations the data comes from are shown on the map below.

1.2 问题分析

我们发现该Assignment一共分位四个部分

1. 首先记录2005-2014年每一天的最高气温和最低气温，这需要对时间数据进行pd.to_datetime转化后拆分然后利用分组函数groupby和聚合函数求最大最小值就可以了。得到每一天的数据后根据日期画出折线图，将最高温度和最低温之间的部分填充上色。
2. 找到2015年超过2005-2014年最高温度和低于最低温度的日期和温度，在图上用散点图表示，可以在plt.scatter()函数中利用np.where()来当index实现，np.where()返回0和1的矩阵。
3. 将2月29日的数据剔除
4. 做好可视化，减少图像垃圾，例如，减少无关数据的笔墨。

二、具体代码及注释

2.1 代码

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
%matplotlib notebook
binsize=400
hashid='fb441e62df2d58994928907a91895ec62c2c42e6cd075c2700843b89'

#读取数据
df=pd.read_csv('data/C2A2_data/BinnedCsvs_d{}/{}.csv'.format(binsize,hashid))
#df=pd.read_csv('assignment2_data.csv')

#温度单位转化
df['value']=df['Data_Value'].apply(lambda x:x/10)

#拆分时间
df['year']=pd.to_datetime(df['Date']).apply(lambda x:x.year)
df['month']=pd.to_datetime(df['Date']).apply(lambda x:x.month)
df['day']=pd.to_datetime(df['Date']).apply(lambda x:x.day)

#去除2月29日的数据
df=df[~((df['month']==2)&(df['day']==29))]

#取2005-2014的数据为df_05_14
df_05_14=df[(df['year']>=2005)&(df['year']<=2014)]
#取2015年的数据为df_15
df_15=df[df['year']==2015]

#取05-14年数据最大值和最小值
df_max_05_14=df_05_14[df_05_14['Element']=='TMAX'].groupby(['month','day']).agg({'value':np.max})
df_min_05_14=df_05_14[df_05_14['Element']=='TMIN'].groupby(['month','day']).agg({'value':np.min})

#取15年数据最大值和最小值
df_max_15=df_15[df_15['Element']=='TMAX'].groupby(['month','day']).agg({'value':np.max})
df_min_15=df_15[df_15['Element']=='TMIN'].groupby(['month','day']).agg({'value':np.min})

#找到打破记录的日期
broken_max=np.where(df_max_15>df_max_05_14)[0]
broken_min=np.where(df_min_15<df_min_05_14)[0]

2.2 绘图结果