cs231n assignment knn

CS231n assignment KNN k_nearest_neighbor.py

K_Nearest Neighbor Classfier
概述：就是单纯比对图片像素距离，在实际中很少用。
1）During training:就是先’记忆‘（仅存储）大量图片
2）During testing, kNN classifies every test image by comparing to all training images and transfering the labels of the k most similar training example
具体分两步：
a)就是计算测试图片与每一个’训练‘图片 距离
b)得出距离后，对于每一张图片，选出前k个 距离最近的图片，然后在这k个图片进行"投票”， 例如k=3,若这三张图片中有两张属于猫，这测试的图片就预测为猫。
3）The value of k is cross-validated（交叉验证）， 选出合适的k值。

数据

X_train, y_train, X_test, y_test = load_CIFAR10(cifar10_dir)
X_train, X_test 分别是’训练‘和 测试的图片数据。y_train, y_test 分别是训练所对应的图片类型，测试所用的图片类型，用0-9表示分类```

K_nearest_neighbor.py

class KNearestNeighbor(object):
    """ a kNN classifier with L2 distance """

    def __init__(self):
        pass

    def train(self, X, y):
        """
        Train the classifier. For k-nearest neighbors this is just
        memorizing the training data.

        Inputs:
        - X: A numpy array of shape (num_train, D) containing the training data
          consisting of num_train samples each of dimension D.
        - y: A numpy array of shape (N,) containing the training labels, where
             y[i] is the label for X[i].
        """
        self.X_train = X
        self.y_train = y

    def predict(self, X, k=1, num_loops=0):
        """
        Predict labels for test data using this classifier.

        Inputs:
        - X: A numpy array of shape (num_test, D) containing test data consisting
             of num_test samples each of dimension D.
        - k: The number of nearest neighbors that vote for the predicted labels.
        - num_loops: Determines which implementation to use to compute distances
          between training points and testing points.

        Returns:
        - y: A numpy array of shape (num_test,) containing predicted labels for the
          test data, where y[i] is the predicted label for the test point X[i].
        """
        if num_loops == 0:
            dists = self.compute_distances_no_loops(X)
        elif num_loops == 1:
            dists = self.compute_distances_one_loop(X)
        elif num_loops == 2:
            dists = self.compute_distances_two_loops(X)
        else:
            raise ValueError('Invalid value %d for num_loops' % num_loops)

        return self.predict_labels(dists, k=k)

train 函数就是简单的记忆训练的图片对应的类型

下面三个函数是利用三种方法来求距离矩阵

 def compute_distances_two_loops(self, X):
        """
        Compute the distance between each test point in X and each training point
        in self.X_train using a nested loop over both the training data and the
        test data.

        Inputs:
        - X: A numpy array of shape (num_test, D) containing test data.

        Returns:
        - dists: A numpy array of shape (num_test, num_train) where dists[i, j]
          is the Euclidean distance between the ith test point and the jth training
          point.
        """
        num_test = X.shape[0]
        num_train = self.X_train.shape[0]
        dists = np.zeros((num_test, num_train))
        for i in range(num_test):
            for j in range(num_train):
                #####################################################################
                # TODO:                                                             #
                # Compute the l2 distance between the ith test point and the jth    #
                # training point, and store the result in dists[i, j]. You should   #
                # not use a loop over dimension, nor use np.linalg.norm().          #
                #####################################################################
                # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
                dists[i][j] = np.sqrt(np.sum(np.square(self.X_train[j,:] - X[i,:])))
                pass

                # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
        return dists

dists[i,j] 是表示第i 个测试图片 到 第 j 个训练图片的 Euclidean distance。

 def compute_distances_one_loop(self, X):
        """
        Compute the distance between each test point in X and each training point
        in self.X_train using a single loop over the test data.

        Input / Output: Same as compute_distances_two_loops
        """
        num_test = X.shape[0]
        num_train = self.X_train.shape[0]
        dists = np.zeros((num_test, num_train))
        for i in range(num_test):
            #######################################################################
            # TODO:                                                               #
            # Compute the l2 distance between the ith test point and all training #
            # points, and store the result in dists[i, :].                        #
            # Do not use np.linalg.norm().                                        #
            #######################################################################
            # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)****
            dists[i,:]=np.sqrt(np.sum(np.square(self.X_train-X[i,:]),axis=1))
            pass

            # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
        return dists

matrix.shape[0] 即是行数，num_test = X.shape[0]， 一个图片在矩阵中被转换成一行数据， 故行数就是图片的个数

dists[i,:]=np.sqrt(np.sum(np.square(self.X_train-X[i,:]),axis=1)) 故得到第i行的数据。dists[i,j] 表示第 i 个测试图片到第j个训练图片的距离，dists[i,:] 表示第 i 个测试图片到所有训练图片的距离

def compute_distances_no_loops(self, X):
dists = np.sqrt(np.multiply(np.dot(X, self.X_train.T), -2) +
                        np.sum(self.X_train ** 2, axis=1) +
                        np.sum(X ** 2, axis=1)[:, np.newaxis])

第三种方法有点难度。 若有两个向量 a,b, 则a,b间的距离 (a-b)2 = aa - 2 * a.dot(b) + bb。 第三种方法就是有这样的思想，但需要注意矩阵大小。

predit_labels 函数

 def predict_labels(self, dists, k=1):
        """
        Given a matrix of distances between test points and training points,
        predict a label for each test point.

        Inputs:
        - dists: A numpy array of shape (num_test, num_train) where dists[i, j]
          gives the distance betwen the ith test point and the jth training point.

        Returns:
        - y: A numpy array of shape (num_test,) containing predicted labels for the
          test data, where y[i] is the predicted label for the test point X[i].
        """
        num_test = dists.shape[0]
        y_pred = np.zeros(num_test)
        for i in range(num_test):
            # A list of length k storing the labels of the k nearest neighbors to
            # the ith test point.
            closest_y = []
            #########################################################################
            # TODO:                                                                 #
            # Use the distance matrix to find the k nearest neighbors of the ith    #
            # testing point, and use self.y_train to find the labels of these       #
            # neighbors. Store these labels in closest_y.                           #
            # Hint: Look up the function numpy.argsort.                             #
            #########################################################################
            # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
            closest_y = self.y_train[np.argsort(dists[i,:])[:k]]
            pass

            # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
            #########################################################################
            # TODO:                                                                 #
            # Now that you have found the labels of the k nearest neighbors, you    #
            # need to find the most common label in the list closest_y of labels.   #
            # Store this label in y_pred[i]. Break ties by choosing the smaller     #
            # label.                                                                #
            #########################################################################
            # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
            y_pred[i] = np.argmax(np.bincount(closest_y))
            pass

            # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****

        return y_pred

closest_y = self.y_train[np.argsort(dists[i,:])[:k]]

argsort()函数是将x中的元素从小到大排列，提取其对应的index(索引号)```
后面中括号[:k]是前k 个的意思， dists[i,j] 是表示第i 个测试图片 到 第 j 个训练图片的 Euclidean distance。则np.argsort(dists[i,:])[:k] 是dists 矩阵i 行上前k个小值所对应的j（index), and use self.y_train to find the labels of these neighbors. 然后找到出现次数最多的那个label.!

y_pred[i] = np.argmax(np.bincount(closest_y)) 用来找到出现最多的那个label.这就是第i 个测试图片的预测label
numpy.argmax(numpy.bincount(dlabel)) returns the most common value found in dlabel

np.bincount(x), 表示从0到9（数组最大值）出现的次数
np.argmax() returns the index of the (first) maximum value in an array。