the complexity of solving a l0-norm optimization problem

the solution for a sparse recovery problem is given by:

min||x||0 min||x||0

s.ty=Ax s.ty=Ax

The definition of  ||x||0 ||x||0 is no. of non-zero entries in  x x. This is also called the sparsity of the vector.

i.e., we are asking for the sparsest solution  x x, that satisfies  y=Ax y=Ax.

Consider the simplest case where  x x is 1-sparse but you don't know the location of that non-zero entry. In such a case, we have  (N1) (N1) possibilities for a 1-sparse vector and to find the solution we have to examine the values of all the  (N1) (N1) possibilities for finding the unique minimizer. Similarly, if you are told that  x x is  k k-sparse, you need to search  (Nk) (Nk) possibilities. i.e., the algorithm grows as  (Nk) (Nk) with increase in  k k. Since  k k is not known a priori, you have to check for all the  N N possible values of  k k.

Hence, the complexity of the algorithm is  ∑i=1N(Ni) ∑i=1N(Ni). This is called as NP hard problem which is an acronym for Non-polynomial time complexity.