Gradient和Hill-climbing算法的测试

Gradient和Hill-climbing算法的测试，课程作业

Assignment Part B

Task 1: Gradient ascent and Hill-climbing with a simple function

Q1: About the question, we need to enter the random position multiple times to obtain the global optimal value and the number of iterations to obtain the average value.

And here are the parameters of the two algorithms:

        Grad Ascent: LRate = 0.1, NumSteps = 50

        Hillclimb: MaxMutate = 1, NumSteps = 50

Through the enumeration of x, y, and the calculation of functions, we can get the conclusion that in Simple Landscape, the global optimal value is 4. So, in the program, if the two values are reached, it is considered that the optimal value is reached and the calculation is finished.

Through 10000 random position tests about Simple Landscape, the following results can be obtained:

        In Grad Ascent, the average number of times to reach the best position is 21.61708434384263. And the probability of reaching the best position is 0.37106289371062895

        In Hillclimb, the average number of times to reach the best position is 24.823821673156974. And the probability of reaching the best position is 0.7446255374462554

Both algorithms are greedy algorithms for local selection, but the difference is that the step size in the Grad Ascent algorithm is fixed, while the step size in the Hillclimb algorithm is obtained by the heuristic method. Therefore, the disadvantage of the two methods is that they are easy to fall into the local optimal solution and can not necessarily search for the global optimal solution.

F.1 Simple Landscape        F.2 Fall local optimal solution   F.3 Reach the optimal value

 

As can be seen from the results, under the current value, Hillclimb needs more iterations to reach the optimal value, but its stability is better. I think the reason for this phenomenon is due to the parameter setting.

 

Q2 and Q3:For this question, we will select 200 points in the range of X from -2 to 2 and 200 points in the range of Y from -2 to 2 to calculate whether the optimal solution can be reached by substituting 200*200 values into the algorithm.

In Grad Ascent(The point in the yellow area indicates that the optimal solution can be obtained, while the purple area cannot）:

    From the principle of the grad ascent algorithm, it can be known that if the ascent rate (now is LRate) is too small, the selected points are more likely to fall into the local optimal situation, and at the same time, more steps are required to reach the optimal solution. Through the test of different rates, in the interval of the rate of 0.1 to 0.16, with the change of rate, the distribution of points to reach the optimal solution does not change, and the average number of steps taken is reduced. In the interval of the rate from 0.17 to 0.25, the points that can obtain the optimal solution increase, and the average number of steps is reduced. In the end, all points can reach the optimal solution except for the point in the special position (point with x equal to 0). But when the rate increases to a certain value (for example 0.5 and 0.6), there is another point where the optimal solution cannot be obtained.

    I plotted the values in the test into line graphs, and the above conclusions can be easily seen from them. Besides, there are some thoughts about the data obtained from the test.

First, when x is 0, no matter the value of Y or the value of LRate, the optimal solution cannot be reached. This is because the ascending gradient calculated on this line is 0. In addition to the point in a special position, other points will participate in the discussion later. In the initial assumption, with the increase of LRate, the number of points that can reach the optimal solution will increase until all points can be reached. However, when LRate = 0.5 and 0.6 are tested, the results are different, and the point of falling into the local optimal solution appears again. This is because the learning rate is too high, which leads to the oscillation near the saddle point.

    Then from the function image, the optimal solution is on the far right, that is, when x = 2 and y = 2, so with the increase of LRate, the number of steps to reach the optimal solution is also decreasing. However, when LRate values are 0.18 and 0.19, the average number of steps there is higher than at LRate = 0.17. This is a very abnormal phenomenon, but I don't know the cause of it.

In Hillclimb:

    From the data obtained from the test, it can be found that whether the Hillclimb algorithm can obtain the best solution is related to the mutation rate (MaxMutate). When the mutation rate increases, there will be more points that can obtain the optimal solution, and the step size will decrease. At the same time, when the mutation rate is lower than 1, the point near x=0 is more difficult to obtain the optimal solution than other points, because x=0 is the location of the saddle point. The Hillclimb algorithm in a simple function is simpler than the Grad Ascent algorithm. But please note that this does not mean that the Hillclimb algorithm is better.

   In the simple function, we can find that the performance of the gradient rising algorithm is better when comparing the best performance of the two functions. Grad Ascent assures you to optimize a continuous function and you can compute its gradient in a given state.

Task 2: Gradient ascent and Hill-climbing with a complex function

Q1: In Complex Landscape the global optimal value is about 12.235（Because the highest value is difficult to reach in a complex function, it is approximately 12）.

        F.1 Complex Landscape            F.2 Projection in y and z directions

Randomly selected 100,000 points to test in a complex function(The parameters used are the same as in Task1Q1)H, and obtained the following results：

 

        In Grad Ascent, the average number of times to reach the best position is 24.092751285411836. And the probability of reaching the best position is 0.0991890081099189

        In Hillclimb, the average number of times to reach the best position is 0. And the probability of reaching the best position is 0.00034999650003499967

 

Obviously, the probability of reaching the optimal solution in a complex function drops significantly. There are some points in the Grad Ascent algorithm that can finally reach the optimal solution through iteration, but the point to reach the optimal solution in the Hillclimb algorithm is that the initial position is generated around the optimal solution.

 

Q2: Since the upper limit of the number of iterations in the random test is 50, it is difficult to obtain the optimal solution, and which will be changed in the next traversal test.

    Change the upper limit of the number of iterations to 100(NumSteps=100) and perform a 200*200 traversal on the range of x and y from -3 to 7, that is, in the area of 10 * 10.

It can be seen from the image that in the GradAscent algorithm, points near the optimal solution are more likely to obtain the optimal solution, and some areas are impossible to reach. This situation is caused by falling into a local optimal solution and oscillating in some positions.

If we reduce the ascent rate to 0.01, then only the area near the optimal solution can reach the optimal situation, and other points will fall into the local optimal. This is very similar to when the ascent rate value is 0.08, so I guess that when the ascent rate is lower than a certain value, the processing capacity for the local optimal situation will be very low, even to the point where it can only handle the optimal solution.

In the figure, we can see that in the region where y is from -3 to 1, and x is from 1 to 3, the change with the mutation rate is small. Almost all of them can reach the optimal solution. This is because it is near the optimal solution. As the mutation rate increases, some points in this region cannot reach the optimal solution, but some new points in other regions leave the local optimal solution and reach the optimal solution.

 

If we change the number of iterations to 50 and 1000, the resulting image will look like this.

By comparing with 100 iterations, it is found that the gap between 50 iterations is the largest, while there is no obvious gap between 1000 iterations. Therefore, I don't think a larger number of iterations is of great help because after a certain number of operations, all the points have reached the stable region. When the number of iterations reaches a stable position, the defect is obvious.

In addition, in the later experiments, I found that with the increase in the number of points and the number of iterations, the time spent by the program doubled. I think we can speed up the experiment process by matrix operation and function optimization.