矩阵分解的Jungle
美帝的有心人士收集了市面上的矩阵分解的几乎所有算法和应用,由于源地址在某神秘物质之外,特转载过来,源地址
Matrix Decompositions
The sources for this list include
Most of the algorithms listed below generally rely on using the nuclear norm as a proxy to the rank functional.
In terms of notations, A refers to a matrix, L refers to a low rank matrix, S a sparse one and N to a noisy one. This page lists the different codes that implement the following matrix factorizations:
Matrix Completion, A = H.*L with H a known mask,
The idea of this approach is to complete the unknown coefficients of a matrix based on the fact that the matrix is low rank:
- OptSpace:
Matrix Completion from a Few Entries by Raghunandan H. Keshavan, Andrea Montanari, and Sewoong Oh - LMaFit: Low-Rank Matrix Fitting
- **
Penalty Decomposition Methods for Rank Minimization by Zhaosong Lu and Yong Zhang.The attendant MATLAB code is here. - Jellyfish:
Parallel Stochastic Gradient Algorithms for Large-Scale Matrix Completion, B. Recht, C. Re, Apr 2011 - GROUSE: Online Identification and Tracking of Subspaces from Highly Incomplete Information, L. Balzano, R. Nowak, B. Recht, 2010
- SVP:
Guaranteed Rank Minimization via Singular Value Projection, R. Meka, P. Jain, I.S.Dhillon, 2009 - SET: SET: an algorithm for consistent matrix completion, W. Dai, O. Milenkovic, 2009
- NNLS: An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems, K. Toh, S. Yun, 2009
- FPCA: Fixed point and Bregman iterative methods for matrix rank minimization, S. Ma, D. Goldfard, L. Chen, 2009
- SVT: A singular value thresholding algorithm for matrix completion, J-F Cai, E.J. Candes, Z. Shen, 2008
Noisy
- GoDec
: Randomized Low-rank and Sparse Matrix Decomposition in Noisy Case - ReProCS: The
Recursive Projected Compressive Sensing code (example)
Robust PCA : A = L + S
- Robust PCA
: Two Codes that go with the paper “Two Proposals for Robust PCA Using Semidefinite Programming.” by MichaleI Mccoy and Joel Tropp - SPAMS (SPArse Modeling Software)
- ADMM:
Alternating Direction Method of Multipliers ‘‘Fast Automatic Background Extraction via Robust PCA’ by Ivan Papusha. The poster is here. The matlab implementation is here. - PCP:
Generalized Principal Component Pursuit - Augmented Lagrange Multiplier (ALM) Method [exact ALM - MATLAB
zip] [inexact ALM - MATLABzip], Reference - The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices, Z. Lin, M. Chen, L. Wu, and Y. Ma (UIUC Technical Report UILU-ENG-09-2215, November 2009) - Accelerated Proximal Gradient , Reference -
Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix, Z. Lin, A. Ganesh, J. Wright, L. Wu, M. Chen, and Y. Ma (UIUC Technical Report UILU-ENG-09-2214, August 2009)[full SVD version - MATLAB zip] [partial SVD version - MATLAB zip] - Dual Method [MATLAB
zip], Reference - Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix, Z. Lin, A. Ganesh, J. Wright, L. Wu, M. Chen, and Y. Ma (UIUC Technical Report UILU-ENG-09-2214, August 2009). - Singular Value Thresholding [MATLAB
zip]. Reference - A Singular Value Thresholding Algorithm for Matrix Completion, J. -F. Cai, E. J. Candès, and Z. Shen (2008). - Alternating Direction Method [MATLAB
zip] , Reference - Sparse and Low-Rank Matrix Decomposition via Alternating Direction Methods, X. Yuan, and J. Yang (2009). - LMaFit: Low-Rank Matrix Fitting
- Bayesian robust PCA
- Compressive-Projection PCA
(CPPCA)
Sparse PCA:
Sparse PCA
- R. Jenatton, G. Obozinski, F. Bach. Structured Sparse Principal Component Analysis. International Conference on Artificial Intelligence and Statistics (AISTATS). [pdf] [code]
- SPAMs
- DSPCA:
Sparse PCA using SDP . Code is here. - PathPCA: A fast greedy algorithm for Sparse PCA. The code is
here.
Dictionary Learning: A = DX
Some implementation of dictionary learning implement the NMF
- Online Learning for Matrix Factorization and Sparse Coding
by Julien Mairal, Francis Bach, Jean Ponce,Guillermo Sapiro [The code is released as SPArse Modeling Softwareor SPAMS] - Dictionary Learning Algorithms for Sparse Representation
(Matlab implementation of FOCUSS/FOCUSS-CNDL is here) - Multiscale sparse image representation with learned dictionaries
[Matlab implementation of the K-SVD algorithm is here, a newer implementation by Ron Rubinstein is here ] - Efficient sparse coding algorithms
[ Matlab code is here ] - Shift Invariant Sparse Coding of Image and Music Data. Matlab implemention is
here - Shift-invariant dictionary learning for sparse representations: extending K-SVD.
- Thresholded Smoothed-L0 (SL0) Dictionary Learning for Sparse Representations
by Hadi Zayyani, Massoud Babaie-Zadeh and Remi Gribonval. - Non-negative Sparse Modeling of Textures (NMF)
[Matlab implementation of NMF (Non-negative Matrix Factorization) and NTF (Non-negative Tensor), a faster implementation of NMF can be found here, here is a more recent Non-Negative Tensor Factorizations package]
NMF: A = DX with
Non-negative Matrix Factorization (NMF) on wikipedia
- HALS:
Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization by Nicolas Gillis, François Glineur. - SPAMS (SPArse Modeling Software)
by Julien Mairal, Francis Bach, Jean Ponce,Guillermo Sapiro - NMF:
C.-J. Lin. Projected gradient methods for non-negative matrix factorization. Neural Computation, 19(2007), 2756-2779. - Non-Negative Matrix Factorization:
This page contains an optimized C implementation of the Non-Negative Matrix Factorization (NMF) algorithm, described in [Lee & Seung 2001]. We implement the update rules that minimize a weighted SSD error metric. A detailed description of weighted NMF can be found in[Peers et al. 2006]. - NTFLAB for Signal Processing, Toolboxes for NMF (Non-negative Matrix Factorization) and NTF (Non-negative Tensor Factorization) for BSS (Blind Source Separation)
- Non-negative Sparse Modeling of Textures (NMF)
[Matlab implementation of NMF (Non-negative Matrix Factorization) and NTF (Non-negative Tensor), a faster implementation of NMF can be found here, here is a more recent Non-Negative Tensor Factorizations package]
Multiple Measurement Vector (MMV) Y = A X with
- T-MSBL/T-SBL
by Zhilin Zhang - Compressive MUSIC with optimized partial support for joint sparse recovery
by Jong Min Kim, Ok Kyun Lee, Jong Chul Ye [no code] - The REMBO Algorithm Accelerated Recovery of Jointly Sparse Vectors
by Moshe Mishali and Yonina C. Eldar [ no code]
Blind Source Separation (BSS) Y = A X with
Include Independent Component Analysis (ICA), Independent Subspace Analysis (ISA), and Sparse Component Analysis (SCA).
ICA:
- ICALab:
http://www.bsp.brain.riken.jp/ICALAB/ - BLISS softwares:
http://www.lis.inpg.fr/pages_perso/bliss/deliverables/d20.html - MISEP:
http://www.lx.it.pt/~lbalmeida/ica/mitoolbox.html - Parra and Spence’s frequency-domain convolutive ICA:http://people.kyb.tuebingen.mpg.de/harmeling/code/convbss-0.1.tar
- C-FICA:
http://www.ast.obs-mip.fr/c-fica
SCA:
- DUET:
http://sparse.ucd.ie/publications/rickard07duet.pdf (the matlab code is given at the end of this pdf document) - LI-TIFROM:
http://www.ast.obs-mip.fr/li-tifrom
Randomized Algorithms
These algorithms uses generally random projections to shrink very large problems into smaller ones that can be amenable to traditional matrix factorization methods.
Resource
Randomized algorithms for matrices and data
Randomized Algorithms for Low-Rank Matrix Decomposition
- Randomized PCA
- Randomized Least Squares:
Blendenpik( http://pdos.csail.mit.edu/~petar/papers/blendenpik-v1.pdf )
Other factorization
D(T(.)) = L + E
- RASL: Robust Batch Alignment of Images by Sparse and Low-Rank Decomposition
- TILT: Transform Invariant Low-rank Textures
Frameworks featuring advanced Matrix factorizations
For the time being, few have integrated the most recent factorizations.
- Scikit Learn
(Python) - Matlab Toolbox for Dimensionality Reduction
(Probabilistic PCA, Factor Analysis (FA)…) - Orange
(Python) - pcaMethods—a
bioconductor package providing PCA methods for incomplete data. R Language
GraphLab / Hadoop
- Danny Bickson
keeps a blog on GraphLab.
Books
Example of use
- CS: Low Rank Compressive Spectral Imaging and a multishot CASSI
- CS: Heuristics for Rank Proxy and how it changes everything….
- Tennis Players are Sparse !
Sources
Arvind Ganesh’s
- Raghunandan H. Keshavan‘ s
list - Stephen Becker’s list
- Nuclear Norm and Matrix Recovery
through SDP by Christoph Helmberg - Nuit Blanche
Relevant links
Reference:
A Unified View of Matrix Factorization Models by Ajit P. Singh and Geoffrey J. Gordon