cs231n-2017-assignments2-TensorFlow.ipynb 数据分析

最新推荐文章于 2022-05-14 19:39:51 发布
最新推荐文章于 2022-05-14 19:39:51 发布
阅读量1.3k 收藏
点赞数
分类专栏： tensorflow 文章标签： tensorflow deep learning cs231n data analysis
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/u013269921/article/details/75831453
版权

今天 自动调参程序跑完了，得到了所有的results，现将数据分析方法及结论记录：

原始数据如下：

lr = 0.005 decay = 0.99 momentum = 0 is_reg = 1 stop@Epoch 18,  train_accuracy = 0.969 val_acc = 0.835 test_acc = 0.822
lr = 0.005 decay = 0.99 momentum = 0 is_reg = 0 stop@Epoch 28,  train_accuracy = 0.981 val_acc = 0.826 test_acc = 0.825
lr = 0.005 decay = 0.99 momentum = 0.1 is_reg = 1 stop@Epoch 17,  train_accuracy = 0.965 val_acc = 0.814 test_acc = 0.81
lr = 0.005 decay = 0.99 momentum = 0.1 is_reg = 0 stop@Epoch 17,  train_accuracy = 0.965 val_acc = 0.829 test_acc = 0.821
lr = 0.005 decay = 0.99 momentum = 0.2 is_reg = 1 stop@Epoch 12,  train_accuracy = 0.945 val_acc = 0.813 test_acc = 0.809
lr = 0.005 decay = 0.99 momentum = 0.2 is_reg = 0 stop@Epoch 15,  train_accuracy = 0.96 val_acc = 0.818 test_acc = 0.812
lr = 0.005 decay = 0.9 momentum = 0 is_reg = 1 stop@Epoch 11,  train_accuracy = 0.944 val_acc = 0.805 test_acc = 0.791
lr = 0.005 decay = 0.9 momentum = 0 is_reg = 0 stop@Epoch 15,  train_accuracy = 0.962 val_acc = 0.817 test_acc = 0.79
lr = 0.005 decay = 0.9 momentum = 0.1 is_reg = 1 stop@Epoch 13,  train_accuracy = 0.954 val_acc = 0.807 test_acc = 0.805
lr = 0.005 decay = 0.9 momentum = 0.1 is_reg = 0 stop@Epoch 13,  train_accuracy = 0.951 val_acc = 0.819 test_acc = 0.808
lr = 0.005 decay = 0.9 momentum = 0.2 is_reg = 1 stop@Epoch 18,  train_accuracy = 0.967 val_acc = 0.796 test_acc = 0.819
lr = 0.005 decay = 0.9 momentum = 0.2 is_reg = 0 stop@Epoch 20,  train_accuracy = 0.97 val_acc = 0.82 test_acc = 0.817
lr = 0.005 decay = 0.8 momentum = 0 is_reg = 1 stop@Epoch 13,  train_accuracy = 0.949 val_acc = 0.82 test_acc = 0.811
lr = 0.005 decay = 0.8 momentum = 0 is_reg = 0 stop@Epoch 15,  train_accuracy = 0.957 val_acc = 0.805 test_acc = 0.807
lr = 0.005 decay = 0.8 momentum = 0.1 is_reg = 1 stop@Epoch 12,  train_accuracy = 0.944 val_acc = 0.811 test_acc = 0.795
lr = 0.005 decay = 0.8 momentum = 0.1 is_reg = 0 stop@Epoch 16,  train_accuracy = 0.962 val_acc = 0.825 test_acc = 0.815
lr = 0.005 decay = 0.8 momentum = 0.2 is_reg = 1 stop@Epoch 15,  train_accuracy = 0.954 val_acc = 0.827 test_acc = 0.815
lr = 0.005 decay = 0.8 momentum = 0.2 is_reg = 0 stop@Epoch 12,  train_accuracy = 0.941 val_acc = 0.798 test_acc = 0.803
lr = 0.001 decay = 0.99 momentum = 0 is_reg = 1 stop@Epoch 19,  train_accuracy = 0.981 val_acc = 0.826 test_acc = 0.833
lr = 0.001 decay = 0.99 momentum = 0 is_reg = 0 stop@Epoch 20,  train_accuracy = 0.981 val_acc = 0.841 test_acc = 0.834
lr = 0.001 decay = 0.99 momentum = 0.1 is_reg = 1 stop@Epoch 19,  train_accuracy = 0.98 val_acc = 0.838 test_acc = 0.828
lr = 0.001 decay = 0.99 momentum = 0.1 is_reg = 0 stop@Epoch 12,  train_accuracy = 0.964 val_acc = 0.839 test_acc = 0.825
lr = 0.001 decay = 0.99 momentum = 0.2 is_reg = 1 stop@Epoch 16,  train_accuracy = 0.976 val_acc = 0.834 test_acc = 0.819
lr = 0.001 decay = 0.99 momentum = 0.2 is_reg = 0 stop@Epoch 27,  train_accuracy = 0.985 val_acc = 0.831 test_acc = 0.831
lr = 0.001 decay = 0.9 momentum = 0 is_reg = 1 stop@Epoch 22,  train_accuracy = 0.983 val_acc = 0.822 test_acc = 0.827
lr = 0.001 decay = 0.9 momentum = 0 is_reg = 0 stop@Epoch 13,  train_accuracy = 0.971 val_acc = 0.833 test_acc = 0.823
lr = 0.001 decay = 0.9 momentum = 0.1 is_reg = 1 stop@Epoch 20,  train_accuracy = 0.981 val_acc = 0.833 test_acc = 0.821
lr = 0.001 decay = 0.9 momentum = 0.1 is_reg = 0 stop@Epoch 19,  train_accuracy = 0.982 val_acc = 0.814 test_acc = 0.815
lr = 0.001 decay = 0.9 momentum = 0.2 is_reg = 1 stop@Epoch 19,  train_accuracy = 0.982 val_acc = 0.846 test_acc = 0.827
lr = 0.001 decay = 0.9 momentum = 0.2 is_reg = 0 stop@Epoch 10,  train_accuracy = 0.956 val_acc = 0.827 test_acc = 0.822
lr = 0.001 decay = 0.8 momentum = 0 is_reg = 1 stop@Epoch 14,  train_accuracy = 0.971 val_acc = 0.825 test_acc = 0.817
lr = 0.001 decay = 0.8 momentum = 0 is_reg = 0 stop@Epoch 14,  train_accuracy = 0.972 val_acc = 0.826 test_acc = 0.809
lr = 0.001 decay = 0.8 momentum = 0.1 is_reg = 1 stop@Epoch 17,  train_accuracy = 0.977 val_acc = 0.839 test_acc = 0.826
lr = 0.001 decay = 0.8 momentum = 0.1 is_reg = 0 stop@Epoch 18,  train_accuracy = 0.977 val_acc = 0.828 test_acc = 0.814
lr = 0.001 decay = 0.8 momentum = 0.2 is_reg = 1 stop@Epoch 11,  train_accuracy = 0.961 val_acc = 0.822 test_acc = 0.815
lr = 0.001 decay = 0.8 momentum = 0.2 is_reg = 0 stop@Epoch 17,  train_accuracy = 0.977 val_acc = 0.813 test_acc = 0.81
lr = 0.0005 decay = 0.99 momentum = 0 is_reg = 1 stop@Epoch 22,  train_accuracy = 0.984 val_acc = 0.834 test_acc = 0.833
lr = 0.0005 decay = 0.99 momentum = 0 is_reg = 0 stop@Epoch 12,  train_accuracy = 0.966 val_acc = 0.835 test_acc = 0.832
lr = 0.0005 decay = 0.99 momentum = 0.1 is_reg = 1 stop@Epoch 14,  train_accuracy = 0.972 val_acc = 0.809 test_acc = 0.824
lr = 0.0005 decay = 0.99 momentum = 0.1 is_reg = 0 stop@Epoch 14,  train_accuracy = 0.974 val_acc = 0.816 test_acc = 0.827
lr = 0.0005 decay = 0.99 momentum = 0.2 is_reg = 1 stop@Epoch 10,  train_accuracy = 0.954 val_acc = 0.824 test_acc = 0.813
lr = 0.0005 decay = 0.99 momentum = 0.2 is_reg = 0 stop@Epoch 13,  train_accuracy = 0.967 val_acc = 0.838 test_acc = 0.832
lr = 0.0005 decay = 0.9 momentum = 0 is_reg = 1 stop@Epoch 26,  train_accuracy = 0.987 val_acc = 0.837 test_acc = 0.831
lr = 0.0005 decay = 0.9 momentum = 0 is_reg = 0 stop@Epoch 18,  train_accuracy = 0.981 val_acc = 0.831 test_acc = 0.828
lr = 0.0005 decay = 0.9 momentum = 0.1 is_reg = 1 stop@Epoch 17,  train_accuracy = 0.98 val_acc = 0.846 test_acc = 0.83
lr = 0.0005 decay = 0.9 momentum = 0.1 is_reg = 0 stop@Epoch 12,  train_accuracy = 0.968 val_acc = 0.832 test_acc = 0.825
lr = 0.0005 decay = 0.9 momentum = 0.2 is_reg = 1 stop@Epoch 13,  train_accuracy = 0.972 val_acc = 0.842 test_acc = 0.832
lr = 0.0005 decay = 0.9 momentum = 0.2 is_reg = 0 stop@Epoch 13,  train_accuracy = 0.971 val_acc = 0.85 test_acc = 0.832
lr = 0.0005 decay = 0.8 momentum = 0 is_reg = 1 stop@Epoch 17,  train_accuracy = 0.978 val_acc = 0.824 test_acc = 0.818
lr = 0.0005 decay = 0.8 momentum = 0 is_reg = 0 stop@Epoch 19,  train_accuracy = 0.981 val_acc = 0.836 test_acc = 0.819
lr = 0.0005 decay = 0.8 momentum = 0.1 is_reg = 1 stop@Epoch 19,  train_accuracy = 0.981 val_acc = 0.844 test_acc = 0.825
lr = 0.0005 decay = 0.8 momentum = 0.1 is_reg = 0 stop@Epoch 18,  train_accuracy = 0.98 val_acc = 0.836 test_acc = 0.822
lr = 0.0005 decay = 0.8 momentum = 0.2 is_reg = 1 stop@Epoch 15,  train_accuracy = 0.975 val_acc = 0.822 test_acc = 0.816
lr = 0.0005 decay = 0.8 momentum = 0.2 is_reg = 0 stop@Epoch 16,  train_accuracy = 0.977 val_acc = 0.826 test_acc = 0.833
使用 EXCEL-2016 的 数据 功能，将上述数据导入，得到表格，在按照stop@Epoch升序或者test_acc降序，找到epoch较小，且test_acc较高的hyperparameter：

如下表中，第2行，第5、6、7均为表现较好的hyperparameter，

lr=0.001decay=0.9momentum=0.2is_reg=0stop@Epoch10train_accuracy=0.956val_acc=0.827test_acc=0.822
lr=0.0005decay=0.99momentum=0is_reg=0stop@Epoch12train_accuracy=0.966val_acc=0.835test_acc=0.832
lr=0.001decay=0.99momentum=0.1is_reg=0stop@Epoch12train_accuracy=0.964val_acc=0.839test_acc=0.825
lr=0.0005decay=0.9momentum=0.1is_reg=0stop@Epoch12train_accuracy=0.968val_acc=0.832test_acc=0.825
lr=0.0005decay=0.99momentum=0.2is_reg=0stop@Epoch13train_accuracy=0.967val_acc=0.838test_acc=0.832
lr=0.0005decay=0.9momentum=0.2is_reg=1stop@Epoch13train_accuracy=0.972val_acc=0.842test_acc=0.832
lr=0.0005decay=0.9momentum=0.2is_reg=0stop@Epoch13train_accuracy=0.971val_acc=0.85test_acc=0.832
lr=0.001decay=0.9momentum=0is_reg=0stop@Epoch13train_accuracy=0.971val_acc=0.833test_acc=0.823
lr=0.0005decay=0.99momentum=0.1is_reg=0stop@Epoch14train_accuracy=0.974val_acc=0.816test_acc=0.827
lr=0.0005decay=0.99momentum=0.1is_reg=1stop@Epoch14train_accuracy=0.972val_acc=0.809test_acc=0.824
lr=0.0005decay=0.8momentum=0.2is_reg=0stop@Epoch16train_accuracy=0.977val_acc=0.826test_acc=0.833
lr=0.0005decay=0.9momentum=0.1is_reg=1stop@Epoch17train_accuracy=0.98val_acc=0.846test_acc=0.83
lr=0.001decay=0.8momentum=0.1is_reg=1stop@Epoch17train_accuracy=0.977val_acc=0.839test_acc=0.826
lr=0.005decay=0.99momentum=0.1is_reg=0stop@Epoch17train_accuracy=0.965val_acc=0.829test_acc=0.821
lr=0.0005decay=0.9momentum=0is_reg=0stop@Epoch18train_accuracy=0.981val_acc=0.831test_acc=0.828
lr=0.005decay=0.99momentum=0is_reg=1stop@Epoch18train_accuracy=0.969val_acc=0.835test_acc=0.822
lr=0.0005decay=0.8momentum=0.1is_reg=0stop@Epoch18train_accuracy=0.98val_acc=0.836test_acc=0.822
lr=0.001decay=0.99momentum=0is_reg=1stop@Epoch19train_accuracy=0.981val_acc=0.826test_acc=0.833
lr=0.001decay=0.99momentum=0.1is_reg=1stop@Epoch19train_accuracy=0.98val_acc=0.838test_acc=0.828
lr=0.001decay=0.9momentum=0.2is_reg=1stop@Epoch19train_accuracy=0.982val_acc=0.846test_acc=0.827
lr=0.0005decay=0.8momentum=0.1is_reg=1stop@Epoch19train_accuracy=0.981val_acc=0.844test_acc=0.825
lr=0.001decay=0.99momentum=0is_reg=0stop@Epoch20train_accuracy=0.981val_acc=0.841test_acc=0.834
lr=0.001decay=0.9momentum=0.1is_reg=1stop@Epoch20train_accuracy=0.981val_acc=0.833test_acc=0.821
lr=0.0005decay=0.99momentum=0is_reg=1stop@Epoch22train_accuracy=0.984val_acc=0.834test_acc=0.833
lr=0.001decay=0.9momentum=0is_reg=1stop@Epoch22train_accuracy=0.983val_acc=0.822test_acc=0.827
lr=0.0005decay=0.9momentum=0is_reg=1stop@Epoch26train_accuracy=0.987val_acc=0.837test_acc=0.831
lr=0.001decay=0.99momentum=0.2is_reg=0stop@Epoch27train_accuracy=0.985val_acc=0.831test_acc=0.831
lr=0.005decay=0.99momentum=0is_reg=0stop@Epoch28train_accuracy=0.981val_acc=0.826test_acc=0.825


结论：

 1.加入momentum有利于快速收敛

2.其实并不需要所谓的自动调参？手工调几个结果也差不多(╥╯^╰╥)：还是说对于这个project不需要，而以后的其他project需要

3.上述results及分析并不完善，比如：

1）所有上述results的model中均有dropout，是否是因为在有dropout的情况下，导致上述results中regularization效果不明显？

2）为何在decay=0.8时，表现明显变差？

3）未完待续

魔方QaQ
关注
专栏目录
cs231n-2017-assignments2-TensorFlow.ipynb 心得体会
魔方QaQ的博客
07-21 2429
新手菜鸟级别，大神请绕道 今天较好的完成了cs231n-2017-assignments2-TensorFlow.ipynb中的最后一个部分：构建一个model对cifar数据集进行训练，今天的精度在train上达到0.98，在val上达到0.82，明天继续进行优化。 具体代码见TensorFlow.ipynb 以下仅贴出model： def my_model(X,y,is_trainin
CS231n Assignment2 tensorflow.ipynb
coldplayplay的博客
03-23 2358
1. 前言经过前期自己写全连接层(FC)，归一化层(Batch Normalization)，Relu层，Pool层，softmax以及svm的loss层，（还没有写dropout层，以及conv层），对Deeplearning有了更清晰的了解。终于来到深度学习架构的学习，考虑到以后工作可能不是搞学术研究（偏学术的推荐pytorch，大佬推荐），选择了跨平台性好的tensorflow。主要是写ten
数据分析处理.ipynb
m0_54186398的博客
05-19 446
# -*- coding: utf-8 -*- # @Time : 2021/5/19 # @Author : Henrette_L import os import pandas as pd import time start = time.time() path = '/Users/eason/Documents/PythonProgrames/数据分析/Henriette/data/' path = input("请输入文件路径：") filename = input("请输入文件保存名
cs231n'18：Assignment 2 | TensorFlow
FortiLZ的博客
06-29 840
Assignment 2 | TensorFlow 感觉 TensorFlow 比 PyTorch 难上手多了，最受不了的是建 graph 的时候没法调试，必须把一个 session 跑起来才能看到数据的形状，这岂不是背离了 Python 写到哪运行到哪里的初衷？先暂时放弃 TensorFlow，主攻 PyTorch 去了。...
CS231n - Assignment2 Tensorflow
donghai_yu的博客
05-01 2016
本次的作业很贴心，在ipython的作业中有一段教程大概告诉我们tensorflow的基本使用，还附上了一些常用API的guide链接，赞！没有科学上网也没有关系，我这里分享一个API查询神器 DevDocs，offline data对于我这种家里没有网的同志更是实用性MAX！作业过程中碰到不清楚的API随手一查即可~作业代码地址：my_github废话不多说，看作业首先是作业里已经写好的simp...
CSE-231N-AI_assignments:该存储库包含CSE 231N编程任务的解决方案-人工智能-IIT（BHU）瓦拉纳西
05-16
CSE-231N-AI_作业 该存储库包含CSE 231N-人工智能-IIT（BHU）瓦拉纳西编程任务的解决方案。 以下分配已添加到资源库中 作业1-使用Viterbi算法进行POS标记 实施维特比算法和隐马尔可夫模型，通过使用训练数据计算过渡...
CS231NPPT及Assignments1-3
08-28
《深度学习视觉识别：斯坦福CS231n课程精华》 斯坦福大学的CS231n课程是由李飞飞教授主讲的一门深度学习与计算机视觉的精品课程，深受全球学者和工程师的欢迎。这门课程的核心是通过PPT讲解和实际的编程作业...
CS231：CS231n：用于视觉识别的卷积神经网络的完整分配
02-06
CS231n分配解决方案 完成分配 2017年Spring。 我刚刚完成在线课程，此仓库包含我对作业的解决方案！ 深入学习深度学习的好地方。 非常感谢CS231斯坦福大学的所有小伙子！ 查找课程笔记和作业并确保查看和 视频选讲...
CS-380-assignments
03-22
【压缩包子文件的文件名称列表】"CS-380-assignments-main" 这个文件名可能表示这是整个作业集的主目录或者起始点，可能包含了所有作业的目录结构，或者是所有作业的汇总文件。打开这个文件，我们可能会看到各个作业...
斯坦福CS231n作业代码（汉化）Assignment 2 Q5
BigDataDigest的博客
02-06 4714
TensorFlow是个什么东东？ 编写：土豆 MoreZheng SlyneD 校对：碧海听滔 Molly 总校对与审核：寒小阳 在前面的作业中你已经写了很多代码来实现很多的神经网络功能。Dropout, Batch Norm 和 2D卷积是深度学习在计算机视觉中的一些重活。你已经很努力地让你的代码有效率以及向量化。 对于这份作业的最后一个部分，我们不会继续探讨之前的代码，而
cs231n-assigment2完整代码
07-09
斯坦福cs231n的assignment2，完整代码
CS231n assignment2
03-12
斯坦福CS231n assignment2整个代码 基于python，利用numpy编写两层神经网络。
cs231n-Assignment-2 python.
12-27
CS231n的Assignment, 用Python3.6.3写的，大家可以参考一下，
斯坦福CS231课程全套资料.zip
05-12
斯坦福大学李飞飞CS231课程 包括课程PPT 英文原版笔记 中文翻译完整版笔记 并且都带书签！！！
阿里云天池龙珠计划—Python训练营： Task 4 Python数据分析：从0完成一个数据分析实战——.ipynb文件分享
NewComerSyt的博客
06-26 315
天池的训练营的内容还不错，介绍了在使用Python进行机器学习中会用到的常见语法。但对于0基础学习Python的读者而言，部分地方可能需要稍加注释进行进一步解释，为此我上传了我的学习.ipynb文件，与大家分享。 如有错误，欢迎指出。 该Task是天池上面一个比赛：链接 点我下载.ipynb文件 ...
CS231n Assignment2--Q3
lovefreedom22的博客
03-07 537
Q3: DropoutDropout.ipynbX_val: (1000, 3, 32, 32) X_train: (49000, 3, 32, 32) X_test: (1000, 3, 32, 32) y_val: (1000,) y_train: (49000,) y_test: (1000,)Dropout forward passRunning tests with
数据分析笔记
Black_713的博客
05-14 272
数据分析 数据分析环境搭建 自己搭建开发环境 安装python（自定义目录） 配置环境变量，python ，pip(Script) 数据分析环境 numpy,pandas,jupyter,scipy,sklean,matplotlib等 启动jupyter: jupyter notebook 映射到浏览器中 使用Anaconde集成开发环境 安装Anaconde(自定义安装目录) 配置环境变量，python（Anaconde） ，pip(Script) 启动jupyter: jupyter noteb
CS231n-2017 第8讲 使用PyTorch
suredied的博客
11-06 767
一. PyTorch文档连接 Tutorial: https://pytorch.org/tutorials/beginner/former_torchies_tutorial.html Layers: https://pytorch.org/docs/nn.html Activations: https://pytorch.org/docs/nn.html#non-linear-activat...
cs231n-assignment3的笔记
Zamirquito的博客
04-03 2126
......Q1: Image Captioning with Vanilla RNNs (25 points)首先是rnn_step_forward，直接按照公式即可：next_h = np.tanh(x.dot(Wx) + prev_h.dot(Wh) + b) # [N, H] cache = (x, prev_h, Wx, Wh, b, next_h)rnn_step_backward，...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值