【Python】数据分析 Section 1.1: Introduction | from Coursera “Applied Data Science with Python“

1. The Python Programming Language: Functions

x=1
y=2
x+y

>>> 3

y

>>> 2

add_numbers is a function that takes two numbers and adds them together.

def add_numbers(x, y):
        return x+y

add_numbers(1, 2)
>>> 3

add_numbers updated to take an optional 3rd parameter. Using print allows printing of multiple expressions within a single cell. 

def add_numbers(x, y, z=None):
        if (z==None):
                return x+y
        else:
                return x+y+z
print(add_numbers(1, 2))
print(add_numbers(1, 2, 3))

>>> 3
>>> 6

add_numbers updated to take an optional flag parameter.

def add_numbers(x, y, z=None, flag=False):
    if (flag):
        print('Flag is true!')
    if (z == None):
        return x + y
    else:
        return x + y + z

print(add_numbers(1, 2, flag=True))

>>> Flag is true!
>>> 3

assign function add_numbers to variable a.

def add_numbers(x, y):
    return x + y

a = add_numbers
a(1, 2)

>>> 3

2. The Python Programming Language: Types and Sequences

Use type to return the object's type.

type('This is a string')

>>> str

type(None)

>>> NoneType

type(1)

>>> int

type(1.0)

>>> float

type(add_numbers)

>>> function

Tuples are an immutable data structure (cannot be altered).

x = (1, 'a', 2, 'b')
type(x)

>>> tuple

Lists are a mutable data structure.

x=[1, 'a', 2, 'b']
type(x)

>>> list

Use append to append an object to a list.

x.append(3.3)
print(x)

>>> [1, 'a', 2, 'b', 3.3]

This is an example of how to loop through each item in the list.

for item in x:
        print(item)

>>> 1
>>> a
>>> 2
>>> b
>>> 3.3

 Or using the indexing operator:

i = 0
while (i != len(x)):
    print(x[i])
    i = i + 1

>>> 1
>>> a
>>> 2
>>> b
>>> 3.3

Use + to concatenate lists.

[1, 2] + [3, 4]

>>> [1, 2, 3, 4]

Use * to repeat lists.

[1] * 3

>>> [1, 1, 1]

Use the in operator to check if something is inside a list.

1 in [1, 2, 3]

>>> True

Now let's look at strings. Use bracket notation to slice a string.

x = 'This is a string'
print(x[0])  #first character
print(x[0:1])  #first character, but we have explicitly set the end character
print(x[0:2])  #first two characters

>>> T
>>> T
>>> Th

This will return the last element of the string.

x[-1]

>>> 'g'

This will return the slice starting from the 4th element from the end and stopping before the 2nd element from the end.

x[-4:-2]

>>> 'ri'

This is a slice from the beginning of the string and stopping before the 3rd element.

x[:3]

>>> 'Thi'

And this is a slice starting from the 4th element of the string and going all the way to the end.

x[3:]

>>> 's is a string'

firstname = 'Christopher Arthur Hansen Brooks'.split(' ')[0]  # [0] selects the first element of the list
lastname = 'Christopher Arthur Hansen Brooks'.split(' ')[-1]  # [-1] selects the last element of the list
print(firstname)
print(lastname)

>>> Christopher
>>> Brooks

Make sure you convert objects to strings before concatenating.

'Chris' + str(2)

>>> 'Chris2'

Dictionaries associate keys with values.

x = {'Christopher Brooks': 'brooksch@umich.edu', 'Bill Gates': 'billg@microsoft.com'}
x['Christopher Brooks']  # Retrieve a value by using the indexing operator

>>> 'brooksch@umich.edu'

x['Kevyn Collins-Thompson'] = None
for name in x:
    print(x[name])

>>> brooksch@umich.edu
>>> billg@microsoft.com
>>> None

for email in x.values():
        print(email)

>>> brooksch@umich.edu
>>> billg@microsoft.com
>>> None

for name, email in x.items():
        print(name)
        print(email)

>>> Christopher Brooks
>>> brooksch@umich.edu
>>> Bill Gates
>>> billg@microsoft.com
>>> Kevyn Collins-Thompson
>>> None

You can unpack a sequence into different variables:

x = ('Christopher', 'Brooks', 'brooksch@umich.edu')
fname, lname, email = x
fname

>>> 'Christopher'

lname

>>> 'Brooks'

Make sure the number of values you are unpacking matches the number of variables being assigned.

x = ('Christopher', 'Brooks', 'brooksch@umich.edu', 'Ann Arbor')
fname, lname, email = x

>>> ValueError

3. The Python Programming Language: More on Strings

sales_record = {
    'price': 3.24,
    'num_items': 4,
    'person': 'Chris'}
sales_statement = '{} bought {} item(s) at a price of {} each for a total of {}'
print(sales_statement.format(sales_record['person'],
                             sales_record['num_items'],
                             sales_record['price'],
                             sales_record['num_items'] * sales_record['price']))

>>> Chris bought 4 item(s) at a price of 3.24 each for a total of 12.96

4. Reading and Writing CSV files

Let's import our datafile mpg.csv, which contains fuel economy data for 234 cars.

• mpg : miles per gallon
• class : car classification
• cty : city mpg
• cyl : # of cylinders
• displ : engine displacement in liters
• drv : f = front-wheel drive, r = rear wheel drive, 4 = 4wd
• fl : fuel (e = ethanol E85, d = diesel, r = regular, p = premium, c = CNG)
• hwy : highway mpg
• manufacturer : automobile manufacturer
• model : model of car
• trans : type of transmission
• year : model year

import csv
%precision 2
with open('datasets/mpg.csv') as csvfile:
    mpg = list(csv.DictReader(csvfile))
mpg[:3]  # The first three dictionaries in our list.

>>> 
[{'': '1',
  'manufacturer': 'audi',
  'model': 'a4',
  'displ': '1.8',
  'year': '1999',
  'cyl': '4',
  'trans': 'auto(l5)',
  'drv': 'f',
  'cty': '18',
  'hwy': '29',
  'fl': 'p',
  'class': 'compact'},
 {'': '2',
  'manufacturer': 'audi',
  'model': 'a4',
  'displ': '1.8',
  'year': '1999',
  'cyl': '4',
  'trans': 'manual(m5)',
  'drv': 'f',
  'cty': '21',
  'hwy': '29',
  'fl': 'p',
  'class': 'compact'},
 {'': '3',
  'manufacturer': 'audi',
  'model': 'a4',
  'displ': '2',
  'year': '2008',
  'cyl': '4',
  'trans': 'manual(m6)',
  'drv': 'f',
  'cty': '20',
  'hwy': '31',
  'fl': 'p',
  'class': 'compact'}]

csv.Dictreader has read in each row of our csv file as a dictioinary. Len shows that out list is comprised of 234 dictionaries.

len(mpg)

>>> 234

keys gives us the column names of our csv.

mpg[0].keys()

>>> 
dict_keys(['', 'manufacturer', 'model', 'displ', 'year', 'cyl', 'trans', 'drv', 'cty', 'hwy', 'fl', 'class'])

This is how to find the average cty fuel economy across all cars. All values in the dictionaries are strings, so we need to convert to float.

sum(float(d['cty']) for d in mpg) / len(mpg)

>>> 16.86

Similarly this is how to find the average hwy fuel economy across all cars.

sum(float(d['hwy']) for d in mpg) / len(mpg)

>>> 23.44

Use set to return the unique values for the number of cylinders the cars in out dataset have.

cylinders = set(d['cyl'] for d in mpg)
cylinders

>>> {'4', '5', '6', '8'}

Here's a more complex example where we are grouping the cars by number of cylinder, and finding the average cty mpg for each group.

CtyMpgByCyl = []
for c in cylinders:  # iterate over all the cylinder levels
    summpg = 0
    cyltypecount = 0
    for d in mpg:  # iterate over all dictionaries
        if d['cyl'] == c:  # if the cylinder level type matches,
            summpg += float(d['cty'])  # add the cty mpg
            cyltypecount += 1  # increment the count
    CtyMpgByCyl.append((c, summpg / cyltypecount))  # append the tuple ('cylinder', 'avg mpg')
CtyMpgByCyl.sort(key=lambda x: x[0])
CtyMpgByCyl

>>> 
[('4', 21.01), ('5', 20.50), ('6', 16.22), ('8', 12.57)]

Use set to return the unique values for the class types in our dataset.

vehicleclass = set(d['class'] for d in mpg)  # what are the class types
vehicleclass

>>> 
{'2seater', 'compact', 'midsize', 'minivan', 'pickup', 'subcompact', 'suv'}

And here's an example of how to find the average hwy mpg for each class of vehicle in our dataset.

HwyMpgByClass = []
for t in vehicleclass:  # iterate over all the vehicle classes
    summpg = 0
    vclasscount = 0
    for d in mpg:  # iterate over all dictionaries
        if d['class'] == t:  # if the cylinder amount type matches,
            summpg += float(d['hwy'])  # add the hwy mpg
            vclasscount += 1  # increment the count
    HwyMpgByClass.append((t, summpg / vclasscount))  # append the tuple ('class', 'avg mpg')
HwyMpgByClass.sort(key=lambda x: x[1])
HwyMpgByClass

>>> 
[('pickup', 16.88),
 ('suv', 18.13),
 ('minivan', 22.36),
 ('2seater', 24.80),
 ('midsize', 27.29),
 ('subcompact', 28.14),
 ('compact', 28.30)]

5. The Python Programming Language: Dates and Times

import datetime as dt
import time as tm

time returns the current time in seconds since the Epoch. (January 1st, 1970)

tm.time()

>>> 1714666338.89

Convert the timestamp to datetime.

dtnow = dt.datetime.fromtimestamp(tm.time())
dtnow

>>> datetime.datetime(2024, 5, 2, 16, 12, 19, 849392)

Handy datetime attributes:

dtnow.year, dtnow.month, dtnow.day, dtnow.hour, dtnow.minute, dtnow.second  # get year, month, day, etc.from a datetime

>>> (2024, 5, 2, 16, 12, 19)

timedelta is a duration expressing the difference between two dates.

delta = dt.timedelta(days=100)  # create a timedelta of 100 days
delta

>>> datetime.timedelta(days=100)

date.today returns the current local date.

today=dt.date.today()
today - delta  # the date 100 days ago

>>> datetime.date(2024, 1, 23)

today > today - delta  # compare dates

>>> True

6. The Python Programming Language: Objects and map()

An example of a class in Python:

class Person:
    department = 'School of Information'  #a class variable
    def set_name(self, new_name):  #a method
        self.name = new_name
    def set_location(self, new_location):
        self.location = new_location

person = Person()
person.set_name('Christopher Brooks')
person.set_location('Ann Arbor, MI, USA')
print('{} live in {} and works in the department {}'.format(person.name, person.location, person.department))

>>> Christopher Brooks live in Ann Arbor, MI, USE and works in the department School of Information

Here's an example of mapping the min function between two lists.

store1 = [10.00, 11.00, 12.34, 2.34]
store2 = [9.00, 11.10, 12.34, 2.01]
cheapest = map(min, store1, store2)
cheapest

>>> 
<map at 0x74f204583250>
list(cheapest)

>>> [9.0, 11.0, 12.34, 2.01]

for item in cheapest:
    print(item)

>>> 
9.0
11.0
12.34
2.01

7. The Python Programming Language: Lambda and List Comprehensions

Here's an example of lambda that takes in three parameters and adds the first two.

my_function = lambda a, b, c: a+b
my_function(1, 2, 3)

>>> 3

Let's iterate from 0 to 999 and return the even numbers.

my_list = []
for number in range(0, 1000):
    if number % 2 == 0:
        my_list.append(number)
my_list

>>> 
[0,
 2,
 4,
 6,
 8,
 10,
...

Now the same thing but with list comprehension.

my_list = [number for number in range(0, 1000) if number % 2 == 0]
my_list

>>> 
[0,
 2,
 4,
 6,
 8,
 10,
...

---

Practice 1

Here is a list of faculty teaching this MOOC. Can you write a function and apply it using map() to get a list of all faculty titles and last names (e.g. ['Dr. Brooks', 'Dr. Collins-Thompson', …]) ?

people = ['Dr. Christopher Brooks', 'Dr. Kevyn Collins-Thompson', 'Dr. VG Vinod Vydiswaran', 'Dr. Daniel Romero']

def split_title_and_name(person):
    title = person.split()[0]
    lastname = person.split()[-1]
    return '{} {}'.format(title, lastname)

list(map(split_title_and_name, people))

Practice 2

Here, why don’t you try converting a function into a list comprehension.

def times_tables():
    lst = []
    for i in range(10):
        for j in range (10):
            lst.append(i*j)
    return lst

times_tables() == [j*i for i in range(10) for j in range(10)]

Here’s a harder question which brings a few things together.

Many organizations have user ids which are constrained in some way. Imagine you work at an internet service provider and the user ids are all two letters followed by two numbers (e.g. aa49). Your task at such an organization might be to hold a record on the billing activity for each possible user.

Write an initialization line as a single list comprehension which creates a list of all possible user ids. Assume the letters are all lower case.

lowercase = 'abcdefghijklmnopqrstuvwxyz'
digits = '0123456789'

correct_answer = [a+b+c+d for a in lowercase for b in lowercase for c in digits for d in digits]

correct_answer[:50] # Display first 50 ids