Seaborn 911 : Data Processing and Visulisation with Python (Python Exercise 28)

Data Processing and Visulisation with Python

911 Calls Capstone Project

For this capstone project we will be analyzing some 911 call data from Kaggle. The data contains the following fields:

• lat : String variable, Latitude
• lng: String variable, Longitude
• desc: String variable, Description of the Emergency Call
• zip: String variable, Zipcode
• title: String variable, Title
• timeStamp: String variable, YYYY-MM-DD HH:MM:SS
• twp: String variable, Township
• addr: String variable, Address
• e: String variable, Dummy variable (always 1)

Just go along with this notebook and try to complete the instructions or answer the questions in bold using your Python and Data Science skills!

Data and Setup

Import numpy and pandas

import numpy as np
import pandas as pd

Import visualization libraries and set %matplotlib inline.

import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('whitegrid')
%matplotlib inline

Read in the csv file 911.csv as a dataframe called df

df = pd.read_csv('911.csv')

Check the info() of the df

df.info()

Check the head of df

df.head(3)

Basic Questions

What are the top 5 zipcodes for 911 calls?

df['zip'].value_counts().head()

What are the top 5 townships (twp) for 911 calls?

df['twp'].value_counts().head()

Take a look at the ‘title’ column, how many unique title codes are there?

df['title'].nunique()

df['title']

Creating new features

In the titles column there are “Reasons/Departments” specified before the title code. These are EMS, Fire, and Traffic. Use .apply() with a custom lambda expression to create a new column called “Reason” that contains this string value.

For example, if the title column value is EMS: BACK PAINS/INJURY , the Reason column value would be EMS.

df['Reason'] = df['title'].apply(lambda x :x.split(":")[0])

What is the most common Reason for a 911 call based off of this new column? Show the frequencies for all the reasons.

df['Reason'].value_counts()

Now use seaborn to create a countplot of 911 calls by Reason.

sns.set()
sns.countplot(data=df,x='Reason',palette='viridis');

sns.set()
sns.countplot(data=df,x='Reason');

Now let us begin to focus on time information. What is the data type of the objects in the timeStamp column?

type(df['timeStamp'].iloc[0])

You should have seen that these timestamps are still strings. Use pd.to_datetime to convert the column from strings to DateTime objects.

df['timeStamp'] = pd.to_datetime(df['timeStamp'])

You can now grab specific attributes from a Datetime object by calling them. For example:

time = df[‘timeStamp’].iloc[0]
time.hour

You can use Jupyter’s tab method to explore the various attributes you can call. Now that the timestamp column are actually DateTime objects, use .apply() to create 3 new columns called Hour, Month, and Day of Week. You will create these columns based off of the timeStamp column.

df['Hour'] = df['timeStamp'].apply(lambda x : x.hour)
df['Month'] = df['timeStamp'].apply(lambda x : x.month)
df['DayofWeek'] = df['timeStamp'].apply(lambda x : x.dayofweek)
dmap = {0:'Mon',1:'Tue',2:'Wed',3:'Thu',4:'Fri',5:'Sat',6:'Sun'}
df['DayofWeek'] = df['DayofWeek'].map(dmap)

df.head()

Notice how the Day of Week is an integer 0-6. Use the .map() with this dictionary to map the actual string names to the day of the week:

dmap = {0:'Mon',1:'Tue',2:'Wed',3:'Thu',4:'Fri',5:'Sat',6:'Sun'}

Now use seaborn to create a countplot of the Day of Week column with the hue based off of the Reason column.

sns.set()
sns.countplot(data=df,x='DayofWeek',hue='Reason',palette='viridis');

plt.legend(bbox_to_anchor=(1.05,1),loc=2,borderaxespad=0);

sns.countplot(data=df,x='DayofWeek',hue='Reason')
plt.legend(loc='best');

Now do the same for Month:

sns.set()
sns.countplot(data=df,x='Month',hue='Reason',palette='viridis');

plt.legend(bbox_to_anchor=(1.05,1),loc=2,borderaxespad=0);

sns.countplot(data=df,x='Month',hue='Reason')
plt.legend(loc='best');

Create a new column called ‘Date’ that contains the date from the timeStamp column. You’ll need to use apply along with the .date() method.

df['Date'] = df['timeStamp'].apply(lambda x : x.date())

df.head(3)

Now groupby this Date column with the count() aggregate and create a plot of counts of 911 calls.

df.groupby('Date').count()['twp'].plot(rot=90);

y = df.groupby('Date').count()['lat']
x = df['Date'].unique()
plt.plot(x,y);

Now recreate this plot but create 3 separate plots with each plot representing a Reason for the 911 call

df[df['Reason']=='Traffic'].groupby('Date').count()['twp'].plot(rot=270)
plt.title('Traffic');

df[df['Reason']=='Fire'].groupby('Date').count()['twp'].plot(rot=270)
plt.title('Fire');

df[df['Reason']=='EMS'].groupby('Date').count()['twp'].plot(rot=270)
plt.title('Fire');

Now let’s move on to creating heatmaps with seaborn on our data. We’ll first need to restructure the dataframe so that the columns become the Hours and the Index becomes the Day of the Week. There are lots of ways to do this, but I would recommend trying to combine groupby with an unstack method.

dayHour = df.groupby(by=['DayofWeek','Hour']).count()['Reason'].unstack()  ;   dayHour

m = df.groupby([df['DayofWeek'],df['Hour']])['lat'].count()
pvdata = m.unstack()
pvdata

Now create a HeatMap using this new DataFrame.

fig = plt.figure(figsize=(12,6))
sns.heatmap(dayHour,cmap='viridis');

fig = plt.figure(figsize=(12,6))
sns.heatmap(pvdata);

Now create a clustermap using this DataFrame.

sns.clustermap(dayHour,cmap='viridis');

sns.clustermap(pvdata);

Now repeat these same plots and operations, for a DataFrame that shows the Month as the column.

n = df.groupby([df['DayofWeek'],df['Month']])['lat'].count()
pvdata1 = n.unstack()
pvdata1

fig = plt.figure(figsize=(12,6))
sns.heatmap(pvdata1,cmap='viridis');

fig = plt.figure(figsize=(12,6))
sns.heatmap(pvdata1);

sns.clustermap(pvdata1,cmap='viridis');

sns.clustermap(pvdata1);

Continue exploring the Data however you see fit!