Anatomy of an Optimization Problem

In essence, an optimization has three parts:

1. optimization parameters,
2. a cost function, and
3. (optionally) constraints.

The optimization problem is to find the parameters that minimize the cost function, subject to any constraints given. In math, this is written as

min J(x)

where x are the optimization parameters (a d-dimensional vector),  is the cost function, and      D represents the constraints. If the problem is unconstrained, then D=R . Examples of constraints include those that enforce positivity, or x must satisfy some other set of equations or inequalities.