Classification
Logistic Function(also called sigmoid function) is in the form
which map real number to (0,1) interval.
new form of hypotheses hθ(x) is
and
now h(x) gives probability that our output is 1.
hθ(x)=P(y=1|x;θ)=1−P(y=0|x;θ)
hθ(x)≥0.5→y=1
hθ(x)<0.5→y=0
for example,
z=θ0+θ1x21+θ2x22
where
recap that
now in classification problems
we ues cost function
Cost(hθ(x),y)=−log(hθ(x)) if y = 1
Cost(hθ(x),y)=−log(1−hθ(x)) if y = 0
we can compress cost function into one case
our fully cost function is as follows:
a vector implementation is
the gradient descending algorithm is:
which is coincidently identical to the form of linear regression
and the vectorized implementation is:
Advanced Optimization
use matlab library to implement advanced algorithm
function [jVal, Gradient] = costFunction(theta)
jVal = % write code here to compute J(theta)
gradient = % here to compute derivative J(theta)
end
then we use fminunc()
options = optimset('GradObj', 'on', 'MaxIter', 100);
initialTheta = zeros(2, 1);
[optTheta, functionVal, exitFlag] = fminunc(@costFunction,
initialTheta, options);
Multiclass Classification: One-vs-all
we divide our problem into n+1 binary classification problems; in each one, we predict the probability that ‘y’ is a member of one of our classes.
y∈{1,2,…,n}
h0θ(x)=P(y=0|x;θ)
…
hnθ(x)=P(y=n|x;θ)
prediction=maxi(h(i)θ(x))
over fitting
Underfitting is when the form of our hypothesis function h maps poorly to the trend of the data. It is usually caused by a function that uses too few features.
Overfitting is caused by a hypothesis function that fits the available data but does not generalize well to predict new data. It is usually caused by a complicated function that creates a lot of unnecessary curves and angles unrelated to the data.
Regularization
gradient descent