- 博客(314)
- 收藏
- 关注
RSS订阅原创 A Weight Value Initialization Method for Improving Learning Performance of the Backpropagation Algor
文章目录概主要内容Shimodaira H. and Ltd N. M. C. A weight value initialization method for improving learning performance of the backpropagation algorithm in neural networks.概考虑f(x)=σ(∑i=1nwixi+w0)f(x) = \sigma (\sum_{i=1}^n w_i x_i + w_0)f(x)=σ(i=1∑nwixi
2022-04-02 11:11:37
261
原创 Avoiding False Local Minima by Proper Initialization of Connections
文章目录概主要内容输入层-隐藏层隐藏层-输出层Wessels L. F. A. and Barnard E. Avoiding False local minima by proper initialization of connections. In IEEE Transactions on Neural Networks, 1992.概避免局部最优的一种初始化方法, 文中给出的‘合适的’初始化方法的准则还挺有道理.主要内容本文主要考虑单隐层的情形, 即f(x)=∑j=1Hvjh(∑i=1
2022-03-31 11:43:01
293
原创 Improving the Learning Speed of 2-Layer Neural Networks
文章目录概主要内容一维情形如何加速多维情形代码Nguyen D. and Widrow B. Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. In International Joint Conference on Neural Networks (IJCNN), 1990.概本文提出了一种关于两层网络的权重初始化方法.主要内容
2022-03-25 16:44:05
296
原创 Denoising Diffusion Probabilistic Models (DDPM)
文章目录概主要内容Diffusion modelsreverse processforward process变分界损失求解LtL_{t}LtL0L_0L0最后的算法参数代码Ho J., Jain A. and Abbeel P. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NIPS), 2020.[Page E. Approximating to
2021-12-16 16:02:23
3990
2
原创 Generative Modeling by Estimating Gradients of the Data Distribution
文章目录概主要内容Langevin dynamicsScore MatchingDenoising Score MatchingNoise Conditional Score NetworksSlow mixing of Langevin dynamics损失函数Annealed Langevin dynamics细节代码[Song Y. and Ermon S. Generative modeling by estimating gradients of the data distribution.
2021-12-15 14:31:49
4327
原创 The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
文章目录概主要内容ImageNet-RStreetView StoreFronts (SVSF)DeepFashion RemixedDeepAugment实验结论代码Hendrycks D., Basart S., Mu N., Kadavath S., Wang F., Dorundo E., Desai R., Zhu T., Parajuli S., Guo M., Song D., Steinhardt J. Gilmer J. The many faces of robustness: a
2021-12-11 14:15:55
2459
原创 LTD: Low Temperature Distillation for Robust Adversarial Training
文章目录概主要内容Chen E. and Lee C. LTD: Low temperature distillation for robust adversarial training. arXiv preprint arXiv:2111.02331, 2021.概本文利用distillation来提高网络鲁棒性.主要内容如上图所示, 作者认为, 如果我们用one-hot的标签进行训练, 结果会导致图(b)中的情形, 于是两个分布中间的空袭部分均可以作为分类边界, 从而导致存在大量的对抗样
2021-12-10 11:03:51
247
原创 Faster RCNN
文章目录Girshick R., Donahue J., Darrel T. and Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation Tech report. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014Girshick R. Fast R-CNN. In IEEE
2021-12-08 19:16:52
131
原创 Scalable Rule-Based Representation Learning for Interpretable Classification
文章目录概主要内容Wang Z., Zhang W., Liu N. and Wang J. Scalable rule-based representation learning for interpretable classification. In Advances in Neural Information Processing Systems (NIPS), 2021.概传统的诸如决策树之类的机器学习方法具有很强的结构性, 也因此具有很好的可解释性. 和深度学习方法相比, 这类方法比较难
2021-11-17 18:49:08
584
原创 Mind the Box: $\ell_1$-APGD for Sparse Adversarial Attacks on Image Classifiers
文章目录概主要内容Croce F. and Hein M. Mind the box: ℓ1\ell_1ℓ1-APGD for sparse adversarial attacks on image classifiers. In International Conference on Machine Learning (ICML), 2021.概以往的ℓ1\ell_1ℓ1攻击, 为了保证∥x′−x∥1≤ϵ,x′∈[0,1]d,\|x' - x\|_1 \le \epsilon, x' \
2021-11-16 20:53:43
154
原创 Limitations of the Lipschitz constant as a defense against adversarial examples
文章目录概主要内容Huster T., Chiang C. J. and Chadha R. Limitations of the lipschitz constant as a defense against adversarial examples. In European Conference on Machine Learning and Data Mining (ECML PKDD), 2018.概本文是想说明现有的依赖Lipschitz常数的以获得可验证的鲁棒性存在很大局限性.主要内
2021-11-13 17:22:22
610
原创 Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
文章目录概主要内容深度宽度代码Huang H., Wang Y., Erfani S., Gu Q., Bailey J. and Ma X. Exploring architectural ingredients of adversarially robust deep neural networks. In Advances in Neural Information Processing Systems (NIPS), 2021概本文是对现有的残差网络结构的探索, grid search一个
2021-11-11 19:16:01
285
原创 Helper-Based Adversarial Training
文章目录概主要内容代码Rade R. and Moosavi-Dezfooli S. Helper-based adversarial training: reducing excessive margin to achieve a better accuracy vs. robustness trade-off. In International Conference on Machine Learning (ICML), 2021概本文认为普通的对抗训练会导致不必要的adversarial m
2021-11-10 16:13:09
354
原创 Friendly Adversarial Training
文章目录概主要内容代码Zhang J., Xu X., Han B., Niu G., Cui L., Sugiyama M., Kankanhalli M. Attacks which do not kill training make adversarial learning stronger. In International Conference on Machine Learning (ICML), 2020.概本文提出了一种early-stopped PGD, 通过一种逐渐增强的方法
2021-11-09 19:02:59
850
原创 Understanding and Improving Fast Adversarial Training
文章目录概主要内容Random Step的作用线性性质gradient alignment代码Andriushchenko M. and Flammarion N. Understanding and improving fast adversarial training. In Advances in Neural Information Processing Systems (NIPS), 2020.概本文主要探讨:为什么简单的FGSM不能够提高鲁棒性;为什么FGSM-RS(即加了随机扰
2021-10-23 16:42:11
368
原创 NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
文章目录概主要内容positional encoding额外的细节代码Mildenhall B., Srinivasan P. P., Tancik M., Barron J. T., Ramamoorthi R. and Ng R. NeRF: representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision (ECCV), 2020.概通过MLP
2021-10-10 15:53:57
221
原创 Implicit Neural Representations with Periodic Activation Functions
文章目录概主要内容初始化策略其它的好处Sitzmann V., Martel J. N. P., Bergman A. W., Lindell D. B., Wetzstein G. Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems (NIPS), 2020.概本文提出用sin\sinsin作为激活函数, 并分析
2021-10-07 16:49:49
729
原创 $\infty$-former: Infinite Memory Transformer
文章目录概主要内容如何扩展?实验细节Martins P., Marinho Z. and Martins A. ∞\infty∞-former: Infinite Memory Transformer. arXiv preprint arXiv:2109.00301, 2021.概在transformer中引入一种长期记忆机制.主要内容假设X∈RL×dX \in \mathbb{R}^{L \times d}X∈RL×d, 即每一行xix_ixi代表一个token对应的特征.Attenti
2021-09-26 20:23:26
307
原创 Normalized Cuts and Image Segmentation
文章目录概主要内容求解相似度总的算法流程skimage.future.graph.cutShi J. and Malik J. Normalized cuts and image segmentation. In IEEE Transactions on Pattern Analysis and Machine Intelligence.概在Digital Image Preprocessing的书上看到了这个算法, 对于其公式结果的推出不是很理解, 于是下载下来看了看. 本文主要讲的是一种利用图
2021-09-18 18:20:24
930
原创 SuperPixel
文章目录SLIC Superpixel algorithm距离函数的选择代码Gonzalez R. C. and Woods R. E. Digital Image Processing (Forth Edition).单个像素的意义其实很小, 于是有了superpixel的概念, 即一簇pixels的集合(且这堆pixels共用一个值), 这会导致图片有非常有趣的艺术风格(下图便是取不同的superpixel大小形成的效果, 有种抽象画的感觉?):经过superpixel的预处理后, 图片可以
2021-09-17 20:47:47
1401
原创 Hough Transform
文章目录代码skimage.transform.hough_line本来想偷懒不记录的, 但是这个Hough Transform实在是有趣.通过Canny算法等将edge的大体部分检测了出来, 但是往往这些检测出来的点并不是连续的, 那么怎么才能将这些点合理地连接在一起呢?这个Hough Transform就可以做到这一点. 首先需要明确的一点是, 我们应该将怎么样的点连接起来, 将其中空缺部分的点填补起来? 最简单但是也非常符合直观理解的便是当有多个点处于同一直线的时候, 我们就认为这条线在原图中其
2021-09-16 20:38:03
187
原创 Adversarial Detection methods
文章目录Kernel Density (KD)Local Intrinsic Dimensionality (LID)Gaussian Discriminant Analysis (GDA)Gaussian Mixture Model (GMM)SelectiveNetCombined Abstention Robustness Learning (CARL)Adversarial Training with a Rejection OptionEnergy-based Out-of-distributio
2021-09-10 17:09:02
367
原创 Morphological Image Processing
文章目录概reflection and translationErosion and DilationErosion示例skimage.morphology.erosiondilation示例skimage.morphology.dilation对偶性Opening and ClosingOpening示例skimage.morphology.openingClosing示例skimage.morphology.closing对偶性The Hit-or-Miss Transform一些基本的操作Bounda
2021-09-05 17:13:16
567
原创 OverFeat:Integrated Recognition, Localization and Detection using Convolutional Networks
文章目录概主要内容Sermanet P., Eigen D., Zhang X., Mathieu M., Fergus R., LeCun Y. OverFeat:integrated recognition, localization and detection using convolutional networks. In International Conference on Learning Representations (ICLR), 2014.概通常的sliding window
2021-08-30 19:50:33
162
原创 Fourier Transform
文章目录Fourier Transform基本的定义性质对称性卷积FFTFourier Transform基本的定义严格来说, 傅里叶变换是Schwartz space 上的一一映射, 对于L1L^1L1, 即可积函数我们都可以找到其对应的傅里叶变换.符号定义傅里叶变换: f^(u)\hat{f}(u)f^(u)∫−∞+∞f(t)e−j2πutdt\int_{-\infty}^{+\infty} f(t) e^{-j2\pi u t} \mathrm{d}t∫−∞+∞f(t
2021-08-24 23:42:40
197
原创 Coding
文章目录Coding RedundancyHuffman CodingGolomb CodingArithmetic CodingLZW CodingRun-Length CodingSymbol-Based CodingBit-Plane CodingBlock Transform CodingTransform SelectionSubimage Size SelectionBit AllocationJPEGPredictive CodingOptimal PredictorsOptimal Quan
2021-08-18 17:24:02
200
原创 Wavelet Transforms
文章目录目标小波变换Scaling FunctionsWavelet Functions二者的联系离散的情形高效变换二维的情形示例目标首先, 既然是变换, 那么就是从一个域到另一个域, 即如下:f(x)=∑kcj0(k)φj0,k(x)+∑j=j0∞∑kdj(k)ψj,k(x),cj0=⟨f(x),φj0,k(x)⟩,dj=⟨f(x),ψj,k(x)⟩.f(x) = \sum_k c_{j_0} (k) \varphi_{j_0, k} (x) + \sum_{j=j_0}^{\infty} \su
2021-08-09 22:16:45
139
原创 WHT, SLANT, Haar
文章目录基本酉变换WALSH-HADAMARD TRANSFORMSsequency-ordered WHTSLANT TRANSFORMHaar TransformGonzalez R. C. and Woods R. E. Digital Image Processing (Forth Edition)基本酉变换一维的变换:t=Af,f=AHt,AH=A∗T,AHA=I.\mathbf{t} = \mathbf{A} \mathbf{f}, \\\mathbf{f} = \mathbf
2021-08-04 23:03:05
269
原创 DFT, DHT, DCT, DST
文章目录基本酉变换othersFourier-related TransformsDFTDHTDCT与DFT的联系DST与DFT的联系Gonzalez R. C. and Woods R. E. Digital Image Processing (Forth Edition)基本酉变换一维的变换:t=Af,f=AHt,AH=A∗T,AHA=I.\mathbf{t} = \mathbf{A} \mathbf{f}, \\\mathbf{f} = \mathbf{A}^{H} \mathbf{
2021-08-04 10:57:53
1090
原创 Color Models (RGB, CMY, HSI)
文章目录概定义RGBCMYCMYKHSI相互的转换RGB <=> CMYCMY <=> CMYKCMY > CMYKCMYK > CMYRGB <=> HSIRGB > HSIHSI > RGB代码示例Gonzalez R. C. and Woods R. E. Digital Image Processing (Forth Edition)概除了我们熟悉的RGB模式来表示图片, 还有其他很多种图片表示方式. 其实我现在很想要知道的一点是
2021-07-28 18:31:17
546
原创 Globally-Robust Neural Networks
文章目录概主要内容代码Leino K., Wang Z. and Fredrikson M. Globally-robust neural networks. In International Conference on Machine Learning (ICML), 2021.概本文是一种可验证的鲁棒方法, 并且提出了一种globally-robust的概念, 但是实际看下来并不觉得有特别出彩的地方.主要内容对于网络f:Rn→Rmf : \mathbb{R}^{n} \rightarrow
2021-07-22 18:40:27
236
6
原创 Improving Adversarial Robustness via Channel-Wise Activation Suppressing
文章目录概主要内容代码Bai Y., Zeng Y., Jiang Y., Xia S., Ma X., Wang Y. Improving adversarial robustness via channel-wise activation suppressing. In International Conference on Learning Representations (ICLR), 2021.Yan H., Zhang J., Niu G., Feng J., Tan V., Sugi
2021-07-21 16:57:11
728
原创 Wiener Filtering
文章目录基本滤波的推导特别的情况特别的例子Signals, Systems and Inference, Chapter 11: Wiener Filtering (mit.edu)基本在图像处理的时候, 遇到了这个维纳滤波, 其推导的公式不是很理解, 于是上网查了查, 并做个简单的总结.符号说明x[k]]x[k]]x[k]]观测信号xxx的第k个元素y^\hat{y}y^为yyy的一个估计vvv噪声信号e[k]e[k]e[k]误差, 为e[k]=
2021-07-21 13:29:05
329
原创 Sharpness-Aware Minimization for Efficiently Improving Generalization
文章目录概主要内容代码Foret P., Kleiner A., Mobahi H., Neyshabur B. Sharpness-aware minimization for efficiently improving generalization. In International Conference on Learning Representations.概在训练的时候对权重加扰动能增强泛化性.主要内容如上图所示, 一般的训练方法虽然能够收敛到一个不错的局部最优点, 但是往往这个局
2021-06-30 17:18:34
1700
原创 MLP-Mixer: An all-MLP Architecture for Vision
文章目录概主要内容代码Tolstlkhin I., Houlsby N., Kolesnikov A., Beyer L., Zhai X., Unterthiner T., Yung J., Steiner A., Keysers D., Uszkoreit J., Lucic M., Dosovitskly A. MLP-mixer: an all-mlp architecture for vision. In International Conference on Learning Represe
2021-06-29 17:54:15
220
原创 Reproducing Kernel Hilbert Space (RKHS)
文章目录概主要内容RKHS-wiki概这里对RKHS做一个简单的整理, 之前的理解错得有点离谱了.主要内容首先要说明的是, RKHS也是指一种Hilbert空间, 只是其有特殊的性质.Hilbert空间H\mathcal{H}H, 其中的每个元素f:X→Kf: \mathcal{X} \rightarrow \mathbb{K}f:X→K, 并由内积⟨⋅,⋅,⟩H\langle \cdot, \cdot, \rangle_{\mathcal{H}}⟨⋅,⋅,⟩H建立联系. 我们考虑如下的
2021-06-24 18:18:58
537
原创 Local Relation Networks for Image Recognition
文章目录概主要内容Hu H., Zhang Z., Xie Z., Lin S. Local relation networks for image recognition. In International Conference on Computer Vision (ICCV), 2019.概一种特殊的卷积?主要内容CNN通过许许多多的filters进行模式匹配(a pattern matching process), 非常低效, 本文提出利用局部相关性来替代这些卷积层.输入特征图
2021-06-24 18:18:34
365
原创 Local Relation Networks for Image Recognition
文章目录概主要内容Hu H., Zhang Z., Xie Z., Lin S. Local relation networks for image recognition. In International Conference on Computer Vision (ICCV), 2019.概一种特殊的卷积?主要内容CNN通过许许多多的filters进行模式匹配(a pattern matching process), 非常低效, 本文提出利用局部相关性来替代这些卷积层.输入特征图
2021-06-21 19:32:16
266
原创 iGPT and ViT
文章目录概主要内容iGPTViT代码Chen M., Radford A., Child R., Wu J., Jun H., Dhariwal P., Luan D., Sutskever I. Generative pretraining from pixels. In International Conference on Machine Learning (ICML), 2020.Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D.
2021-06-20 19:47:53
402
原创 GPT and BERT
文章目录概主要内容GPTBERTRadford A., Narasimhan K., Salimans T. and Sutskever I. Improving language understanding by generative pre-training. 2018.Devlin J., Chang M., Lee K. and Toutanova K. BERT: Pre-training of deep bidirectional transformers for language u
2021-06-20 17:44:10
125
空空如也
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人