Jacobi and Guass-Seidel Iteration Update

Background

    Iterative schemes require time to achieve sufficient accuracy and are reserved for large systems of equations where there are a majority of zero elements in the matrix. Often times the algorithms are taylor-made to take advantage of the special structure such as band matrices.  Practical uses include applications in circuit analysis, boundary value problems and partial differential equations.
    
    Iteration is a popular technique finding roots of equations.  Generalization of fixed point iteration can be applied to systems of linear equations to produce accurate results.  The method Jacobi iteration is attributed to Carl Jacobi (1804-1851) and Gauss-Seidel iteration is attributed to  Johann Carl Friedrich Gauss (1777-1855) and Philipp Ludwig von Seidel (1821-1896).

 

    Consider that the n×n square matrix A is split into three parts, the main diagonal D, below diagonal L and above diagonal U.  We have  A = D - L - U.

 =

--  

 

Definition (Diagonally Dominant).  The matrix    is strictly diagonally dominant if

           for   .  

 

Theorem (Jacobi Iteration).  The solution to the linear system    can be obtained starting with  , and using iteration scheme
    
          
where  

          and  .  

If   is carefully chosen a sequence  is generated which converges to the solution  P,  i.e.  .  
A sufficient condition for the method to be applicable is that A is strictly diagonally dominant or diagonally dominant and irreducible.  

Proof  Jacobi and Gauss-Seidel Iteration  Jacobi and Gauss-Seidel Iteration  

 

Theorem (Gauss-Seidel Iteration).  The solution to the linear system    can be obtained starting with  , and using iteration scheme
    
          
where  

          and  .  

If   is carefully chosen a sequence  is generated which converges to the solution  P,  i.e.  .  
A sufficient condition for the method to be applicable is that A is strictly diagonally dominant or diagonally dominant and irreducible.

Proof  Jacobi and Gauss-Seidel Iteration  Jacobi and Gauss-Seidel Iteration  

 

Computer Programs  Jacobi and Gauss-Seidel Iteration  Jacobi and Gauss-Seidel Iteration  

Mathematica Subroutine (Jacobi Iteration).

Mathematica Subroutine (Gauss-Seidel Iteration).

Example 1.  Use Jacobi iteration to solve the linear system  .  
Try 10, 20 and 30 iterations.  
Solution 1.

 

Enter the matrix A, vector B and starting vector P.

Use 10 iterations.

Determine if the method has converged.  

Use 20 iterations.

Determine if the method has converged.  

Use 30 iterations.

Determine if the method has converged.  

 

 

Example 2.  Use Jacobi iteration to attempt solving the linear system  .
Try 10 iterations.  
Observe that something is not working.  In example 5 we will check to see if this matrix is diagonally dominant.   
Solution 2.

 

Example 3.  Use Gauss-Seidel iteration to solve the linear system  .  
Try 10, 20 iterations.  Compare the speed of convergence with Jacobi iteration.
Solution 3.

 

Example 4  Use Gauss-Seidel iteration to attempt solving the linear system  .
Try 10 iterations.  
Observe that something is not working.  In example 5 we will check to see if this matrix is diagonally dominant.   
Solution 4.

 

 

Warning.  

    Iteration does not always converge.  A sufficient condition for iteration to Jacobi iteration to converge is that A is strictly diagonally  dominant. The following subroutine will check to see if a matrix is strictly diagonally dominant.  It should be used before any call to Jacobi iteration or Gauss-Seidel iteration is made.  There exists a counter-example for which Jacobi iteration converges and Gauss-Seidel iteration does not converge.  The "true" sufficient condition for Jacobi iteration to converge is that the "spectral radius" of     is less than 1, where   is the diagonal of  .  That is, the magnitude of the largest eigenvalue of M must be less than 1.  This condition seems harsh because numerical computation of eigenvalues is an advanced topic compared to solution of a linear system.  

Example 5  Test the matrix A of Examples 2 and 4 to see if A is strictly diagonally dominant.
Solution 5.

 

 

More efficient subroutines

    A tolerance can be supplied to either the Jacobi or Gauss-Seidel method which will permit it to exit the loop if convergence has been achieved.  

 

Computer Programs  Jacobi and Gauss-Seidel Iteration  Jacobi and Gauss-Seidel Iteration  

Mathematica Subroutine (Jacobi Iteration).

Mathematica Subroutine (Gauss-Seidel Iteration).

Example 6.  Use Jacobi and Gauss-Seidel iteration to solve the linear system  .  
Use a tolerance of    and a maximum of  50  iterations.
Solution 6.

 

 

Subroutines using matrix commands

    In the Jacobi subroutine we can use fix point iteration as suggested by the theory.  

Mathematica Subroutine (Jacobi Iteration).