目标:Python review + Intro to PySpark
Part 1: Math review
略
Part 2: Numpy
求逆矩阵方法:
# Invert AAt with np.linalg.inv()
AAtInv = np.linalg.inv(AAt)
可参见资料:Numpy
Part 3: Additional NumPy and Spark linear algebra
- 切片和重组
重组方法:hstack(), vstack()
zeros = np.zeros(8)
ones = np.ones(8)
zerosThenOnes = np.hstack((zeros,ones)) # A 1 by 16 array
zerosAboveOnes = np.vstack((zeros,ones)) # A 2 by 8 array
- PySpark’s DenseVector
PySpark用来存放vector的对象
Part 4: Python lambda expressions
NB. lambda 表达式的参数意义一定记清楚
例子,lambda拿来给tuples进行排序:
#two ways, same result
swap1 = lambda x: (x[1],x[0])
swap2 = lambda (x0, x1): (x1, x0)
print 'swap1((1, 2)) = {0}'.format(swap1((1, 2)))
print 'swap2((1, 2)) = {0}'.format(swap2((1, 2)))
相关资料:
lambda tutorial
lambda function
Part 5: CTR data download
略