State the fixed-increment convergence theorem
Let the subject of trainings
X 1
and
X 2
be linearly separable. input vector:
Correspondingly, we define the (p + 1)-by-1 weight vector:
The output is written in the compact form
For fixed n, the equation w T x=0 , devide the inputs into tow classes as devision surface. let X 1 be the subset of training vectors belong to class ξ 1 and let X 2 be the subset of training vectors belong to class ξ 2 . We may state: w T x≥0 for every input vector x belong to class ξ 1 . w T x≤0 for every input vector x belong to class ξ 2 .
Then there exists a weight vector w such that we may state:
1. w T ≥0 for every input vector x belonging to class ξ 1
and
2. w T ≤0 for every input vector x belonging to class ξ 2
The algorithm for adapting the weight vector of the elementary perceptron may now be formulated as follows:
if the
n
th member of the training vector,
1.
w(n+1)=w(n)
if
w T (n)x(n)≥0
and
x
belong to class
ξ 1
2.
w(n+1)=w(n)
if
w T (n)x(n)≤0
and
x
belong to class
ξ 2
otherwise
1.
w(n+1)=w(n)−η(n)x(n)
if
w(n) T x(n)≥0
and
x(n)
belongs to class
ξ 1
2.
w(n+1)=w(n)+η(n)x(n)
if
w(n) T x(n)≤0
and
x(n)
belongs to class
ξ 2
where the learning-rate paramater
η(n)
controls the adjustment applied to the weight vector at iteration
n
.
If
In the sequel,we first prove the convergenece of a fixed inrement adaption rule for which
Prove:
The initial condition
w(0)=0
. Suppose that
w T (n)x(n)<0
for
n=1,2,...
and an input vector
x(n)
belong to the subset
X 1
. So
Given the initial condition w(0)=0 , we may iteratively solve this equation for w(n+1) abtianing the result
As there exists asolution w 0 , we may define a positive number α by the relation
Hence, multiplying both sides of Eq.(2) by the row vector w T 0 , we get
Next, the Cauchy-Schwarz inequality states that
Next, we follow anther development route, as we know (1) rewrite: w(k+1)=w(k)+x(k) , taking the squared Euclidean norm of both sides of Eq.(1), we get
Adding these inequalities for k = 1,…,n, and assuming that the initial condition w=0 , we get the following condition:
We can state that n cannot be larger than some value