CS231n Assignment1 features详解

版权声明:如需转载,请注明出处! https://blog.csdn.net/Fire_to_cheat_/article/details/86772504

Step1 加载数据

from cs231n.features import color_histogram_hsv, hog_feature 

  

def get_CIFAR10_data(num_training=49000, num_validation=1000, num_test=1000): 

    # Load the raw CIFAR-10 data 

    cifar10_dir = 'cs231n/datasets/cifar-10-batches-py' 

  

    X_train, y_train, X_test, y_test = load_CIFAR10(cifar10_dir) 

     

    # Subsample the data 

    mask = list(range(num_training, num_training + num_validation)) 

    X_val = X_train[mask] 

    y_val = y_train[mask] 

    mask = list(range(num_training)) 

    X_train = X_train[mask] 

    y_train = y_train[mask] 

    mask = list(range(num_test)) 

    X_test = X_test[mask] 

    y_test = y_test[mask] 

     

    return X_train, y_train, X_val, y_val, X_test, y_test 

  

# Cleaning up variables to prevent loading data multiple times (which may cause memory issue) 

try: 

   del X_train, y_train 

   del X_test, y_test 

   print('Clear previously loaded data.') 

except: 

   pass 

  

X_train, y_train, X_val, y_val, X_test, y_test = get_CIFAR10_data() 

 

Step2 提取特征

通过HOG(Histogram of Oriented Gradients,方向梯度直方图)提取边缘特征,色彩直方图(使用HSV颜色空间的色彩通道)提取颜色特征,两者共同组成图像的特征向量。

HOG可以捕捉图像的本质特征,色彩直方图可以表示图像的颜色特征,因此将两者结合使用更好。HOG介绍见目标检测的图像特征提取之(一)HOG特征

features.py中的hog_feature函数和color_histogram_hsv函数都是针对单个图像进行计算,并返回该图像的特征向量,extract_features函数可以对多幅图像的多种特征进行提取,将结果存储在矩阵中,该矩阵的每一列存储的是一幅图像所有特征向量。

from cs231n.features import * 

  

num_color_bins = 10 # Number of bins in the color histogram 

feature_fns = [hog_feature, lambda img: color_histogram_hsv(img, nbin=num_color_bins)] 

X_train_feats = extract_features(X_train, feature_fns, verbose=True) 

X_val_feats = extract_features(X_val, feature_fns) 

X_test_feats = extract_features(X_test, feature_fns) 

  

# Preprocessing: Subtract the mean feature 

mean_feat = np.mean(X_train_feats, axis=0, keepdims=True) 

X_train_feats -= mean_feat 

X_val_feats -= mean_feat 

X_test_feats -= mean_feat 


# Preprocessing: Divide by standard deviation. This ensures that each feature  has roughly the same scale. 
std_feat = np.std(X_train_feats, axis=0, keepdims=True) 

X_train_feats /= std_feat 

X_val_feats /= std_feat 

X_test_feats /= std_feat 


# Preprocessing: Add a bias dimension 

X_train_feats = np.hstack([X_train_feats, np.ones((X_train_feats.shape[0], 1))]) 

X_val_feats = np.hstack([X_val_feats, np.ones((X_val_feats.shape[0], 1))]) 

X_test_feats = np.hstack([X_test_feats, np.ones((X_test_feats.shape[0], 1))]) 

 

Step3 在特征上训练SVM

# Use the validation set to tune the learning rate and regularization strength 

  

from cs231n.classifiers.linear_classifier import LinearSVM 

  

learning_rates = [1e-9, 1e-8, 1e-7] 

regularization_strengths = [5e4, 5e5, 5e6] 

  

results = {} 

best_val = -1 

best_svm = None 

  

################################################################################ 

# TODO:                                                                        # 

# Use the validation set to set the learning rate and regularization strength. # 

# This should be identical to the validation that you did for the SVM; save    # 

# the best trained classifer in best_svm. You might also want to play          # 

# with different numbers of bins in the color histogram. If you are careful    # 

# you should be able to get accuracy of near 0.44 on the validation set.       # 

################################################################################ 

for rs in regularization_strengths: 

    for lr in learning_rates: 

        svm = LinearSVM() 

        loss_hist = svm.train(X_train_feats, y_train, lr, rs, num_iters=6000) 

        y_train_pred = svm.predict(X_train_feats) 

        train_accuracy = np.mean(y_train == y_train_pred) 

        y_val_pred = svm.predict(X_val_feats) 

        val_accuracy = np.mean(y_val == y_val_pred) 

        if val_accuracy > best_val: 

            best_val = val_accuracy 

            best_svm = svm            

        results[(lr,rs)] = train_accuracy, val_accuracy 

################################################################################ 

#                              END OF YOUR CODE                                # 

################################################################################ 

  

# Print out results. 

for lr, reg in sorted(results): 

    train_accuracy, val_accuracy = results[(lr, reg)] 

    print('lr %e reg %e train accuracy: %f val accuracy: %f' % ( 

                lr, reg, train_accuracy, val_accuracy)) 

     

print('best validation accuracy achieved during cross-validation: %f' % best_val) 

 

 

运行结果:

lr 1.000000e-09 reg 5.000000e+04 train accuracy: 0.093735 val accuracy: 0.084000 

 
lr 1.000000e-09 reg 5.000000e+05 train accuracy: 0.113061 val accuracy: 0.118000 
 

lr 1.000000e-09 reg 5.000000e+06 train accuracy: 0.415306 val accuracy: 0.422000 
lr 1.000000e-08 reg 5.000000e+04 train accuracy: 0.125939 val accuracy: 0.124000 
lr 1.000000e-08 reg 5.000000e+05 train accuracy: 0.415143 val accuracy: 0.419000 
lr 1.000000e-08 reg 5.000000e+06 train accuracy: 0.409429 val accuracy: 0.411000 
lr 1.000000e-07 reg 5.000000e+04 train accuracy: 0.414653 val accuracy: 0.417000 
lr 1.000000e-07 reg 5.000000e+05 train accuracy: 0.410816 val accuracy: 0.419000 
lr 1.000000e-07 reg 5.000000e+06 train accuracy: 0.360796 val accuracy: 0.374000 
best validation accuracy achieved during cross-validation: 0.422000 

在测试集上预测:

# Evaluate your trained SVM on the test set 

y_test_pred = best_svm.predict(X_test_feats) 

test_accuracy = np.mean(y_test == y_test_pred) 

print(test_accuracy) 

 

结果:

0.422

可视化错误分类

 

# An important way to gain intuition about how an algorithm works is to 

# visualize the mistakes that it makes. In this visualization, we show examples 

# of images that are misclassified by our current system. The first column 

# shows images that our system labeled as "plane" but whose true label is 

# something other than "plane". 

#Describe the misclassification  

examples_per_class = 8 

classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'] 

for cls, cls_name in enumerate(classes): 

    idxs = np.where((y_test != cls) & (y_test_pred == cls))[0] 

    idxs = np.random.choice(idxs, examples_per_class, replace=False) 

    for i, idx in enumerate(idxs): 

        plt.subplot(examples_per_class, len(classes), i * len(classes) + cls + 1) 

        plt.imshow(X_test[idx].astype('uint8')) 

        plt.axis('off') 

        if i == 0: 

            plt.title(cls_name) 

plt.show() 


Step4 在图像特征上训练神经网络

划分数据

# Preprocessing: Remove the bias dimension 

# Make sure to run this cell only ONCE 

print(X_train_feats.shape) 

X_train_feats = X_train_feats[:, :-1] 

X_val_feats = X_val_feats[:, :-1] 

X_test_feats = X_test_feats[:, :-1] 

  

print(X_train_feats.shape) 

 

训练神经网络 

 

from cs231n.classifiers.neural_net import TwoLayerNet 

  

input_dim = X_train_feats.shape[1] 

hidden_dim = 500 

num_classes = 10 

  

net = TwoLayerNet(input_dim, hidden_dim, num_classes) 

best_net = None 

  

################################################################################ 

# TODO: Train a two-layer neural network on image features. You may want to    # 

# cross-validate various parameters as in previous sections. Store your best   # 

# model in the best_net variable.                                              # 

################################################################################ 

# Your code 

results = {} 

best_val = -1 

best_net = None 

  

learning_rates = [1e-2 ,1e-1, 5e-1, 1, 5] 

regularization_strengths = [1e-3, 5e-3, 1e-2, 1e-1, 0.5, 1] 

  

for lr in learning_rates: 

    for reg in regularization_strengths: 

        net = TwoLayerNet(input_dim, hidden_dim, num_classes) 

        # Train the network 

        stats = net.train(X_train_feats, y_train, X_val_feats, y_val, 

        num_iters=1500, batch_size=200, 

        learning_rate=lr, learning_rate_decay=0.95, 

        reg= reg, verbose=False) 

        val_acc = (net.predict(X_val_feats) == y_val).mean() 

        if val_acc > best_val: 

            best_val = val_acc 

            best_net = net          

        results[(lr,reg)] = val_acc 

  

# Print out results. 

for lr, reg in sorted(results): 

    val_acc = results[(lr, reg)] 

    print('lr %e reg %e val accuracy: %f' % ( 

                lr, reg,  val_acc)) 

     

print('best validation accuracy achieved during cross-validation: %f' % best_val) 

################################################################################ 

#                              END OF YOUR CODE                                # 

################################################################################ 

 

 

在测试集上预测结果:

# Run your best neural net classifier on the test set. You should be able 

# to get more than 55% accuracy. 

  

test_acc = (best_net.predict(X_test_feats) == y_test).mean() 

print(test_acc) 

lr 1.000000e-02 reg 1.000000e-03 val accuracy: 0.158000 
 

lr 1.000000e-02 reg 5.000000e-03 val accuracy: 0.195000 
 

lr 1.000000e-02 reg 1.000000e-02 val accuracy: 0.087000 
 

lr 1.000000e-02 reg 1.000000e-01 val accuracy: 0.078000 
 

lr 1.000000e-02 reg 5.000000e-01 val accuracy: 0.087000 
lr 1.000000e-02 reg 1.000000e+00 val accuracy: 0.087000 
lr 1.000000e-01 reg 1.000000e-03 val accuracy: 0.539000 
lr 1.000000e-01 reg 5.000000e-03 val accuracy: 0.526000 
lr 1.000000e-01 reg 1.000000e-02 val accuracy: 0.507000 
lr 1.000000e-01 reg 1.000000e-01 val accuracy: 0.417000 
lr 1.000000e-01 reg 5.000000e-01 val accuracy: 0.105000 
lr 1.000000e-01 reg 1.000000e+00 val accuracy: 0.098000 
lr 5.000000e-01 reg 1.000000e-03 val accuracy: 0.579000 
lr 5.000000e-01 reg 5.000000e-03 val accuracy: 0.581000 
lr 5.000000e-01 reg 1.000000e-02 val accuracy: 0.547000 
lr 5.000000e-01 reg 1.000000e-01 val accuracy: 0.406000 
lr 5.000000e-01 reg 5.000000e-01 val accuracy: 0.087000 
lr 5.000000e-01 reg 1.000000e+00 val accuracy: 0.079000 
lr 1.000000e+00 reg 1.000000e-03 val accuracy: 0.585000 
lr 1.000000e+00 reg 5.000000e-03 val accuracy: 0.555000 
lr 1.000000e+00 reg 1.000000e-02 val accuracy: 0.518000 
lr 1.000000e+00 reg 1.000000e-01 val accuracy: 0.411000 
lr 1.000000e+00 reg 5.000000e-01 val accuracy: 0.174000 
lr 1.000000e+00 reg 1.000000e+00 val accuracy: 0.079000 
lr 5.000000e+00 reg 1.000000e-03 val accuracy: 0.087000 
lr 5.000000e+00 reg 5.000000e-03 val accuracy: 0.087000 
lr 5.000000e+00 reg 1.000000e-02 val accuracy: 0.087000 
lr 5.000000e+00 reg 1.000000e-01 val accuracy: 0.087000 
lr 5.000000e+00 reg 5.000000e-01 val accuracy: 0.087000 
lr 5.000000e+00 reg 1.000000e+00 val accuracy: 0.087000 
best validation accuracy achieved during cross-validation: 0.585000 

 

展开阅读全文

没有更多推荐了,返回首页