神经进化算法——利用NEAT算法解决迷宫导航问题（基于NEAT-Python）

最新推荐文章于 2024-04-26 03:04:19 发布

盼小辉丶

最新推荐文章于 2024-04-26 03:04:19 发布

阅读量8.4k

点赞数 5

分类专栏：深度学习遗传算法文章标签： python 深度学习神经网络遗传算法算法

本文链接：https://blog.csdn.net/LOVEmy134611/article/details/115955686

版权

深度学习同时被 2 个专栏收录

42 篇文章 136 订阅

订阅专栏

遗传算法

12 篇文章 38 订阅

订阅专栏

神经进化算法——利用NEAT算法解决迷宫导航问题（基于NEAT-Python）

迷宫导航问题

迷宫导航问题是经典的计算机问题，与创建可以在模糊环境中找到路径的自主导航智能体密切相关。迷宫环境中，面向目标的适应度函数在迷宫中接近最终目标点的死角中可以具有较大的适应度得分梯度。迷宫的这些区域成为基于目标搜索算法的局部最优解，这些目标算法可能会收敛于这些区域。当搜索算法收敛于这种欺骗性的局部最优时，它找不到合适的迷宫求解器。
在以下示例中，可以看到带有局部最优死角的二维迷宫，如图中阴影所示：
迷宫环境使用基于目标的搜索算法从起点导航到出口的迷宫导航智能体很容易陷入局部最优死角，阻碍基于目标的搜索算法找到成功的迷宫求解器。
假设迷宫中的智能体是配备了一组传感器的机器人，它可以检测附近的障碍物并判断通往迷宫出口的方向。机器人的运动由两个决策器控制，分别用于控制智能体的行驶运动和角运动。机器人的决策器由ANN控制，该ANN接收来自传感器的输入并为决策器产生两个控制信号。

迷宫环境模拟

迷宫模拟环境由三个主要组件构成，这些组件由单独的Python类实现：

Agent：包含与模拟所使用的迷宫导航器智能体相关信息的类（agent.py）。
AgentRecordStore：用于管理演化过程中所有智能体的评估记录的存储类。收集的记录可用于在完成进化过程后对其进行分析（agent.py）。
MazeEnvironment：包含有关迷宫模拟环境信息的类。此类还提供了管理模拟环境、控制求解器智能体的位置、执行碰撞检测以及生成智能体传感器的输入人工神经网络(Artificial Neural Network, ANN)中数据的方法（maze_environment.py）。

接下来，将详细地介绍迷宫模拟环境的每个部分。

迷宫导航智能体

二维迷宫导航任务中，机器人的主要目标是在限定的时间内穿越迷宫到达定义的目标点。控制机器人的ANN是由神经进化算法获得的。
神经进化算法从基本的初始ANN配置开始——该配置只有传感器的输入节点和决策器的输出节点，算法运行过程中ANN配置逐渐变得复杂，直到找到成功的迷宫求解器为止。
下图显示了迷宫导航模拟中使用的迷宫智能体的示意图：
迷宫导航智能体
在上图中，实心圆表示机器人。实心圆中的箭头显示了机器人的前进方向。实心圆周围的六个箭头表示六个测距传感器，这些传感器指示在给定方向上到最近障碍物的距离。四个外圆线段表示四个扇形雷达传感器，它们充当着指向迷宫出口的指南针。
当从目标点到机器人中心的线落入其视场（FOV）内时，特定的雷达传感器将被激活。雷达传感器的检测范围受到落入其FOV的迷宫区域的限制。因此，在任何给定时间，四个雷达传感器之一被激活，指示迷宫出口方向。
雷达传感器相对于机器人的方向具有以下FOV区域：

Sensor	FOV, degrees
Front	315.0~405.0
Left	45.0~135.0
Back	135.0~225.0
Right	225.0~315.0

测距传感器是从机器人中心沿特定方向的射线。当与任何障碍物相交时，它将激活，并返回与障碍物的距离。该传感器的检测范围由特定的配置参数定义。
机器人的测距传感器监视与智能体前进方向相关的以下方向：

Sensor	Direction, degrees
Right	-90.0
Front-right	-45.0
Front	0.0
Front-left	45.0
Left	90.0
Back	-180.0

机器人的运动由两个决策器控制，这两个决策可以更改其速度和角速度。
迷宫导航智能体的Python实现具有多个字段来保存其当前状态并维护其传感器的激活状态：

class Agent:
	def __init__(
	    self,location,heading=0,speed=0,
	    angular_vel=0,radius=8.0,range_finder_range=100.0):
	    self.heading=heading
	    self.speed=speed
	    self.angular_vel=angular_vel
	    self.radius=radius
	    self.range_finder_range=range_finder_range
	    self.location=location
	    # defining the range finder sensors
	    self.range_finder_range = [-90.0,-45.0,0.0,45.0,90.0,-180.0]
	    # defining the radar sensors
	    self.radar_angles = [(315.0,405.0),(45.0,135.0),(135.0,225.0),(225.0,315.0)]
	    # the list to hold range finders activations
	    self.range_finders = [None] * len(self.range_finder_angles)
	    # the list to hold pie-slice radar acivations
	    self.radar = [None] * len(self.radar_angles)

上面的代码显示了Agent类的默认构造函数，其中Agent的所有字段都已初始化。迷宫环境模拟将使用这些字段在每个模拟步骤中存储智能体的当前状态。

迷宫环境实现

为了模拟在迷宫中导航的求解器智能体程序，需要定义一个环境，该环境管理迷宫的配置，跟踪迷宫求解器的位置，并为导航机器人的传感器数据阵列提供输入。
所有这些特性都适合封装到MazeEnvironment Python类中的一个逻辑块，该类包含以下字段：

class MazeEnvironment:
	def __init__(self,agent,walls,exit_point,exit_range=5.0):
	    self.walls = walls
	    self.exit_point=exit_point
	    self.exit_range=exit_range
	    # The maze navigating agent
	    self.agent = agent
	    # The flag to indicate if exit was found
	    self.exit_found = False
	    # The initial distance of agent from exit
	    self.initial_distance = self.agent_distance_to_exit()

上面的代码显示了MazeEnvironment类的默认构造函数及其所有字段的初始化：

迷宫的配置由walls和exit_point确定。墙是线段的列表，每个线段代表迷宫中的一堵墙，而exit_point是迷宫出口的位置。
exit_range字段定义出口区域（exit_point周围的距离范围）的值。当智能体位于出口区域时，意味着成功解决了迷宫问题。
agent字段包含对上一节中描述的已初始化智能体类的引用。
initial_distance字段存储从智能体的起始位置到迷宫出口的距离。此值将用于智能体适应度得分计算。

传感器数据生成

智能体由ANN控制，该ANN需要接收传感器数据作为输入，以产生相应的控制信号作为输出。导航智能体配备了两种传感器的阵列：

六个测距传感器，用于检测与迷宫壁的距离，指示在特定方向上到最近障碍物的距离。
四个扇形雷达传感器，指示迷宫出口点的方向。

传感器值需要在每个模拟步骤中进行更新，并且MazeEnvironment类提供了更新两类传感器的方法。
测距传感器的阵列更新如下：

for i,angle in enumerate(self.agent.range_finder_angles):
    rad = genometry.deg_to_rad(angle)
    projection_point = genometry.Point(
        x = self.agent.location.x + math.cos(rad) * \
            self.agent.range_finder_range,
        y = self.agent.location.y + math.sin(rad) * \
            self.agent.range_finder_range
    )
    projection_point.rotate(
        self.agent.heading,self.agent.location
    )
    projection_line = genometry.Line(
        a = self.agent.location,
        b = self.projection_point
    )
    min_range = self.agent.range_finder_range
    for wall in self.walls:
        found,intersection = wall.intersection(projection_line)
        if found:
            found_range = intersection.distance(self.agent.location)
            if found_range < min_range:
                min_range = found_range
    # store the distance to the closest obstacle
    self.agent.range_finders[i] = min_range

代码枚举了测距传感器的所有检测方向，这些方向由方向角确定（Agent构造函数中的range_finder_angles字段）。然后针对每个方向创建一条投影线，该投影线从智能体的当前位置开始，其长度等于测距仪的检测范围。之后，测试投影线是否与任何迷宫壁相交。如果检测到多个交叉点，则到最近墙的距离将作为值存储到特定的测距传感器。否则，最大检测范围将保存为测距仪传感器的值。
扇形雷达传感器阵列使用MazeEnvironment类中的以下代码进行更新：

    def update_radar(self):
        target = genometry.Point(self.exit_point.x,self.exit_point.y)
        target.rotate(self.agent.heading,self.agent.location)
        target.x -= self.agent.location.x
        target.y -= self.agent.location.y
        angle = target.angle()
        for i,r_angles in enumerate(self.agent.radar_angles):
            self.agent.radar[i] = 0.0 # reset specific radar
            if (angle >= r_angles[0] and angle <= r_angles[1]) or
                (angle + 360 >= r_angles[0] and angle + 360 < r_rangles[1]):
                # fire the radar
                self.agent.radar[i] = 1.0

创建迷宫出口点的副本，并相对于智能体的航向和全局坐标系内的位置旋转迷宫出口点。然后平移目标点以使其与迷宫求解器的局部坐标系对齐；智能体被放置在坐标的原点。之后，计算从坐标的原点到局部坐标系内目标点的矢量角度。该角度是从当前智能体位置到迷宫出口点的方位角。找到方位角后，枚举扇形雷达传感器，以找到在其视场中包含方位角的传感器。通过将其值设置为1来激活相应的雷达传感器，而通过将其值清零来停用其他雷达传感器。

导航智能体

智能体位置更新

从控制器ANN接收到相应的控制信号后，在每个模拟步骤中都需要更新迷宫中智能体的位置。执行以下代码以更新智能体的位置：

    def update(self,control_signals):
        if self.exit_found:
            return True # Maze exit already found
        self.apply_control_signals(control_signals)
        vx = math.cos(genometry.deg_to_rad(self.agent.heading)) * \
            self.agent.speed
        vy = math.sin(genometry.deg_to_rad(self.agent.heading)) * \
            self.agent.speed
        self.agent.heading += self.agent.angular_vel
        if self.agent.heading > 360:
            self.agent.heading -= 360
        elif self.agent.heading < 0:
            self.agent.heading += 360
        new_loc = genometry.Point(
            x = self.agent.location.x + vx,
            y = self.agent.location.y + vy
        )
        if not self.test_wall_collision(new_loc):
            self.agent.location = new_loc
        self.update_rangefinder_sensors()
        self.update_radar()
        distance = self.agent_distance_to_exit()
        self.exit_found = (distance < self.exit_range)
        return self.exit_found

update(self,control_signals)函数在MazeEnvironment类中定义，并在每个模拟时间步中调用。它接收带有控制信号作为输入的列表，并返回一个布尔值，指示迷宫求解器的位置更新后是否已到达出口区域。
此函数开头的代码将接收到的控制信号并应用于更新智能体的速度和线速度，如下所示：

self.agent.angular_vel += (control_signals[0] - 0.5)
self.agent.speed += (control_signals[1] - 0.5)

之后，将计算x和y速度分量以及智能体航向，并将其用于估计其在迷宫中的新位置。如果此新位置不与任何迷宫壁碰撞，则将其分配给智能体作为其当前位置：

 vx = math.cos(genometry.deg_to_rad(self.agent.heading)) * \
            self.agent.speed
        vy = math.sin(genometry.deg_to_rad(self.agent.heading)) * \
            self.agent.speed
        self.agent.heading += self.agent.angular_vel
        if self.agent.heading > 360:
            self.agent.heading -= 360
        elif self.agent.heading < 0:
            self.agent.heading += 360
        new_loc = genometry.Point(
            x = self.agent.location.x + vx,
            y = self.agent.location.y + vy
        )
        if not self.test_wall_collision(new_loc):
            self.agent.location = new_loc

此后，新的智能体位置将用于更新测距和雷达传感器，以估计下一个时间步的新传感器输入：

self.update_rangefinder_sensors()
self.update_radar()

最后，以下函数测试智能体是否已到达迷宫出口：

distance = self.agent_distance_to_exit()
self.exit_found = (distance < self.exit_range)
return self.exit_found

如果已到达迷宫出口，则将exit_found字段的值设置为True以指示任务已成功完成，并且其值从函数调用中返回。

智能体记录存储

完成实验后，评估和可视化智能体在进化过程中各代种群的表现。这是通过在指定时间段内运行迷宫求解模拟后收集有关每个智能体的统计数据来实现的。智能体记录的收集由两个Python类实现：AgentRecord和AgentRecordStore。
从类的构造函数中可以看出，AgentRecord类包含以下数据字段：

    def __init__(self,generation,agent_id):
        self.generation = generation
        self.agent_id = agent_id
        self.x = -1
        self.y = -1
        self.fitness = -1
        self.hit_exit = False
        self.species_id = -1
        self.species_age = -1

字段定义如下：

创建智能体记录时，generation保存世代的ID。
agent_id是唯一的智能体标识符。
x和y是完成模拟后智能体在迷宫中的位置。
fitness是智能体的适应度得分。
hit_exit是一个标志，指示代理是否已到达迷宫出口区域。
species_id和species_age是智能体所属的物种的ID和年龄。

AgentRecordStore类包含一个智能体记录列表，并提供用于将收集的记录从特定文件加载/转储到特定文件的功能。
在评估了基因组适应度之后，将新的AgentRecord实例添加到存储中，评估基因组适应度通过以下代码实现：

def eval_fitness(genome_id,genome,config,time_steps=400):
    maze_env = copy.deepcopy(trialSim.orig_maze_environment)
    control_net = neat.nn.FeedForwardNetwork.create(genome,config)
    fitness = maze.maze_simulation_evaluate(
        env=maze_env,net=control_net,time_steps=time_steps
    )
    record = agent.AgentRecord(
        generation=trialSim.population.generation,
        agent_id=genome_id
    )
    record.fitness = fitness
    record.x = maze_env.agent.location.x
    record.y = maze_env.agent.location.y
    record.hit_exit = maze_env.exit_found
    record.species_id = trialSim.population.species.get_species_id(genome_id)
    record.species_age = record.generation - \
        trialSim.population.species.get_species(genome_id).created
    trialSim.record_store.add_record(record)
    return fitness

该代码首先创建原始迷宫环境的副本，以避免评估运行之间的干扰。之后，它使用提供的NEAT配置从指定的基因组创建ANN，并在给定数量的时间步长下开始迷宫仿真。然后，将返回的智能体适应度得分以及其他统计信息存储到特定的AgentRecord实例中，并将其添加到记录存储中。
将收集的记录将保存到输出目录中的data.pickle文件中，并用于可视化所有评估智能体的性能。

智能体记录可视化

在神经进化过程中收集了所有智能体的评估记录后，可视化记录的数据以观察性能。可视化应包括所有智能体的最终位置，并允许设置物种适应度的阈值，以控制将物种添加到相应的绘图中。

目标函数的定义

该目标函数基于迷宫求解器的适应度得分的估算，该迷宫求解器通过在执行400个模拟步骤后测量其最终位置与迷宫出口之间的距离来进行估算。因此，目标函数是面向目标的，并且仅取决于实验的最终目标：到达迷宫出口区域。
该实验中使用的目标函数确定如下。首先，需要将损失函数定义为模拟结束时智能体的最终位置与迷宫出口的位置之间的欧几里得距离：
$\mathcal L=\sqrt{\sum_{i=1}^2(a_i-b_i)^2}$
$\mathcal L$ 是损失函数， $a$ 是智能体的最终位置坐标， $b$ 是迷宫出口的坐标。
使用上述损失函数，定义适应度函数：
$\mathcal F = \begin{cases} 1.0, & \text{$\mathcal L \le R_{exit}$} \\ \mathcal F_n, & \text{otherwise} \end{cases}$
$R_{exit}$ 是迷宫出口点周围出口区域的半径， $\mathcal F_n$ 是归一化适应度得分。归一化的适应度评分如下：
$\mathcal F_n = {\frac {\mathcal L-D_{init}}{D_{init}}}$
$D_{init}$ 是导航仿真开始时从智能体到迷宫出口的初始距离。
该公式将适应性评分标准化为 $(0, 1]$ ，但在极少数情况下，当智能体的最终位置远离其初始位置并且回合结束时，可能会导致负值。将应用标准化的适应度评分来避免出现负值：
$\mathcal F_n = \begin{cases} 0.01, & \text{$\mathcal F_n \le 0$} \\ \mathcal F_n, & \text{otherwise} \end{cases}$
当适应度得分小于或等于0.01时，将为其分配支持的最小适应度得分值（0.01）；否则，将按原样使用。使最小适应度得分高于零，以使每个基因组都有繁殖的机会。
Python中的以下代码实现了面向目标的目标函数：

# Calculate the fitness score based on distance from exit
fitness = env.agent_distance_to_exit()
if fitness <= self.exit_range:
    fitness = 1.0
else:
    # Normalize fitness to range (0,1]
    fitness = (env.initial_distance - fitness) / \
        env.initial_distance
    if fitness <= 0.01:
        fitness = 0.01

迷宫导航实验

下图显示了用于此实验的迷宫：
迷宫配置
图中的迷宫有两个特定的位置，左上方的圆圈表示迷宫导航智能体的起始位置。右下角的圆圈标记迷宫出口的确切位置。迷宫求解器(智能体)需要到达迷宫出口的附近，该迷宫出口由其周围的特定出口范围区域表示，以完成任务。

NEAT-Python超参数选择

根据目标函数定义，可以到达迷宫出口区域而获得的导航智能体适应度得分的最大值为1.0。将人口规模设置为250。

[NEAT]
fitness_criterion = max
fitness_threshold = 1.0
pop_size = 250
reset_on_extinction = False

表型ANN的初始配置包括10个输入节点，2个输出节点和1个隐藏节点。输入和输出节点对应于输入传感器和控制信号输出。提供隐藏节点是为了从神经进化过程的开始就引入非线性，ANN配置如下：

num_hidden = 1
num_inputs = 10
num_outputs = 2

为了扩展解的搜索范围，需要增加种群的种类，以在有限的世代内尝试不同的基因组构型。这可以通过降低相容性阈值或通过增加用于执行基因组相容性分数计算的系数的值来完成。：

[NEAT]
compatibility_disjoint_coefficient = 1.1
[DefaultSpeciesSet]
compatibility_threshold = 3.0

我们的目标是创建具有最少数量的隐藏节点和连接的迷宫求解器控件ANN。最佳的ANN配置在通过神经进化过程进行训练期间以及在迷宫求解模拟器中进行推理的过程中，在计算上的花费较少。可以通过减少添加新节点的概率来产生最佳的ANN配置：

node_add_prob = 0.1
node_detele_prob = 0.1

最后，允许神经进化过程不仅利用具有前馈连接的ANN配置，而且还利用循环结构。通过使用循环连接，使ANN拥有内存并成为状态机成为可能：

feed_forward = False

迷宫环境配置文件

实验的迷宫环境配置以纯文本格式提供。配置文件的内容如下：

11
30 22
0
270 100
5 5 295 5
295 5 295 135
295 135 5 135
...

迷宫配置文件的格式如下：

第一行包含迷宫中的墙壁数量。
第二行确定智能体的起始位置(x,y)。
第三行表示智能体的初始航向，以度为单位。
第四行显示迷宫出口位置(x,y)。
接下来几行定义了迷宫的墙壁。

迷宫墙由线段表示，前两个数字定义起始端点的坐标，后两个数字确定结束端点的坐标。智能体的起始位置和迷宫出口指示二维空间中某个点的x和y坐标。

实验启动程序

实验启动程序是在maze_experiment.py文件中实现的。提供了以下功能：读取命令行参数，配置和启动神经进化过程，以及呈现实验结果。此外，它还包括回调函数的实现，以评估基因组的适应度。这些回调函数将在初始化期间提供给NEAT-Python库环境。

首先初始化迷宫模拟环境：
```
maze_env_config = os.path.join(local_dir,'%_maze.txt' % args.maze)
maze_env = maze.reade_environment(maze_env_config)
```
args.maze指的是用户在启动Python脚本时提供的命令行参数，指向迷宫环境配置文件。

创建NEAT配置对象，并使用创建的配置对象创建neat.Population对象：

config = neat.Config(neat.DefaultGenome,neat.DefaultReproduction,
    neat.DefaultSpeciesSet,neat.DefaultStagnation,config_file)
p = neat.Population(config)

创建迷宫模拟环境并将其存储为全局变量，以简化适应度评估回调函数对其的访问：
```
global trialSim
trialSim = MazeSimulationTrial(maze_env=maze_env,population=p)
```
MazeSimulationTrial对象包含用于访问原始迷宫模拟环境以及用于保存智能体评估结果的记录存储的字段。在每次对适应度评估回调函数eval_fitness进行调用时，原始迷宫模拟环境将被复制，并由特定的求解器智能体用于迷宫求解模拟。之后，从环境中收集有关智能体的完整统计信息，包括其在迷宫中的最终位置，并将其添加到记录存储中。

添加各种统计报告器：

p.add_record(neat.StdOutReporter(True))
stats = neat.StatisticsReporter()
p.add_reporter(stats)
p.add_reporter(neat.Checkpointer(
    5,filename_prefix='%s/maze-neat-checkpoint-' % trial_out_dir))

报告器可将神经进化过程的中间结果显示在控制台上，并收集统计信息在程序完成后进行可视化。

最后，将神经进化过程运行指定的代数，并检查是否找到了解：

start_time = time.time()
best_genome = p.run(eval_genomes,n=n_generations)
elapsed_time = time.time() - start_time
solution_found = (best_genome.fitness >= config.fitness_threshold)
if solution_found:
    print("SUCCESS: The stable maze solver controller was found!!!")
else:
    print("FAILURE: Failed to find the stable maze solver controller!!!")

假设，如果NEAT-Python库返回的最佳基因组的适应性得分大于或等于配置文件中设置的适应度阈值，则已经找到解。打印完成该过程所花费的时间。

基因组适应度评估

回调函数用于评估属于特定物种的所有基因组的适应度得分：

def eval_genomes(genomes,config):
    for genome_id,genome in genomes:
        genome.fitness = eval_fitness(genome_id,genome,config)

运行迷宫导航实验

执行以下命令，运行试验：

$ python3 maze_experiment.py -m medium -g 150

最后，成功的迷宫求解器控制器ANN的基因组配置打印如下：

 ****** Running generation 218 ****** 

Maze solved in 376 steps
Population's average fitness: 0.26595 stdev: 0.27894
Best fitness: 1.00000 - size: (2, 5) - species 14 - id 53049

Best individual in generation 218 meets fitness threshold - complexity: (2, 5)

Best genome:
Key: 53049
Fitness: 1.0
Nodes:
        0 DefaultNodeGene(key=0, bias=0.19570685490413814, response=1.0, activation=sigmoid, aggregation=sum)
        1 DefaultNodeGene(key=1, bias=-1.6913043338626579, response=1.0, activation=sigmoid, aggregation=sum)
Connections:
        DefaultConnectionGene(key=(-9, 1), weight=-0.2651164185249115, enabled=True)
        DefaultConnectionGene(key=(-8, 1), weight=0.028955000178012114, enabled=True)
        DefaultConnectionGene(key=(-7, 1), weight=2.919475204266212, enabled=True)
        DefaultConnectionGene(key=(-3, 0), weight=2.9321519677683705, enabled=True)
        DefaultConnectionGene(key=(-1, 0), weight=-11.647747360002768, enabled=True)
        DefaultConnectionGene(key=(1, 1), weight=-0.6128097813804334, enabled=False)
SUCCESS: The stable maze solver controller was found!!!

在控制台输出中，可以看到成功的迷宫求解器控制器在演化过程中找到，下图显示了控制器ANN的最终配置：
ANN配置
下图显示了历代求解器智能体的适应度得分：
智能体适应度得分

下图显示了种群的物种分布：

智能体记录可视化

可视化各种物种在进化过程中的表现，可以使用以下命令执行：

$ python3 visualize.py -m medium -r out/maze_objective/medium/data.pickle --width 300 --height 150

该命令加载智能体适应度评估的记录，该记录存储在data.pickle文件中。此后，它在迷宫求解模拟结束时在迷宫上绘制智能体的最终位置。每个智能体的最终位置均以彩色圆圈表示。圆圈的颜色编码特定智能体所属的种类。进化过程中产生的每个物种都有唯一的颜色代码。下图显示了此可视化的结果：
智能体分布
为了使可视化更加有用，引入了适应度阈值以筛选出性能最高的物种。顶部子图显示了属于冠军物种的求解器智能体程序的最终位置（适应性得分高于0.8）。

完整代码

agent.py

import pickle

class Agent:
    """
    This is the maze navigating agent
    """
    def __init__(self, location, heading=0, speed=0, angular_vel=0, radius=8.0, range_finder_range=100.0):
        """
        Creates new Agent with specified parameters.
        Arguments:
            location:               The agent initial position within maze
            heading:                The heading direction in degrees.
            speed:                  The linear velocity of the agent.
            angular_vel:            The angular velocity of the agent.
            radius:                 The agent's body radius.
            range_finder_range:     The maximal detection range for range finder sensors.
        """
        self.heading = heading
        self.speed = speed
        self.angular_vel = angular_vel
        self.radius = radius
        self.range_finder_range = range_finder_range
        self.location = location

        # defining the range finder sensors
        self.range_finder_angles = [-90.0, -45.0, 0.0, 45.0, 90.0, -180.0]

        # defining the radar sensors
        self.radar_angles = [(315.0, 405.0), (45.0, 135.0), (135.0, 225.0), (225.0, 315.0)]

        # the list to hold range finders activations
        self.range_finders = [None] * len(self.range_finder_angles)
        # the list to hold pie-slice radar activations
        self.radar = [None] * len(self.radar_angles)

class AgentRecord:
    """
    The class to hold results of maze navigation simulation for specific
    solver agent. It provides all statistics about the agent at the end
    of navigation run.
    """
    def __init__(self, generation, agent_id):
        """
        Creates new record for specific agent at the specific generation
        of the evolutionary process.
        """
        self.generation = generation
        self.agent_id = agent_id
        # initialize agent's properties
        self.x = -1
        self.y = -1
        self.fitness = -1
        # The flag to indicate whether this agent was able to find maze exit
        self.hit_exit = False
        # The ID of species this agent belongs to
        self.species_id = -1
        # The age of agent's species at the time of recording
        self.species_age = -1

class AgentRecordStore:
    """
    The class to control agents record store.
    """
    def __init__(self):
        """
        Creates new instance.
        """
        self.records = []

    def add_record(self, record):
        """
        The function to add specified record to this store.
        Arguments:
            record: The record to be added.
        """
        self.records.append(record)

    def load(self, file):
        """
        The function to load records list from the specied file into this class.
        Arguments:
            file: The path to the file to read agents records from.
        """
        with open(file, 'rb') as dump_file:
            self.records = pickle.load(dump_file)

    def dump(self, file):
        """
        The function to dump records list to the specified file from this class.
        Arguments:
            file: The path to the file to hold data dump.
        """
        with open(file, 'wb') as dump_file:
            pickle.dump(self.records, dump_file)

geometry

import math

def deg_to_rad(degrees):
    """
    The function to convert degrees to radians.
    Arguments:
        degrees: The angle in degrees to be converted.
    Returns:
        The degrees converted to radians.
    """
    return degrees / 180.0 * math.pi

def read_point(str):
    """
    The function to read Point from specified string. The point
    coordinates are in order (x, y) and delimited by space.
    Arguments:
        str: The string encoding Point coorinates.
    Returns:
        The Point with coordinates parsed from provided string.
    """
    coords = str.split(' ')
    assert len(coords) == 2
    return Point(float(coords[0]), float(coords[1]))

def read_line(str):
    """
    The function to read line segment from provided string. The coordinates
    of line end points are in order: x1, y1, x2, y2 and delimited by spaces.
    Arguments:
        str: The string to read line coordinates from.
    Returns:
        The parsed line segment.
    """
    coords = str.split(' ')
    assert len(coords) == 4
    a = Point(float(coords[0]), float(coords[1]))
    b = Point(float(coords[2]), float(coords[3]))
    return Line(a, b)

class Point:
    """
    The basic class describing point in the two dimensional Cartesian coordinate
    system.
    """
    def __init__(self, x, y):
        """
        Creates new point at specified coordinates
        """
        self.x = x
        self.y = y

    def angle(self):
        """
        The function to determine angle in degrees of vector drawn from the
        center of coordinates to this point. The angle values is in range
        from 0 to 360 degrees in anticlockwise direction.
        """
        ang = math.atan2(self.y, self.x) / math.pi * 180.0
        if (ang < 0.0):
            # the lower quadrants (3 or 4)
            return ang + 360
        return ang

    def rotate(self, angle, point):
        """
        The function to rotate this point around another point with given
        angle in degrees.
        Arguments:
            angle: The rotation angle (degrees)
            point: The point - center of rotation
        """
        rad = deg_to_rad(angle)
        # translate to have another point at the center of coordinates
        self.x -= point.x
        self.y -= point.y
        # rotate
        ox, oy = self.x, self.y
        self.x = math.cos(rad) * ox - math.sin(rad) * oy
        self.y = math.sin(rad) * ox - math.cos(rad) * oy
        # restore
        self.x += point.x
        self.y += point.y

    def distance(self, point):
        """
        The function to caclulate Euclidean distance between this and given point.
        Arguments:
            point: The another point
        Returns:
            The Euclidean distance between this and given point.
        """
        dx = self.x - point.x
        dy = self.y - point.y

        return math.sqrt(dx*dx + dy*dy)
    
    def __str__(self):
        """
        Returns the nicely formatted string representation of this point.
        """
        return "Point (%.1f, %.1f)" % (self.x, self.y)

class Line:
    """
    The simple line segment between two points. Used to represent maze wals.
    """
    def __init__(self, a, b):
        """
        Creates new line segment between two points.
        Arguments:
            a, b: The end points of the line
        """
        self.a = a
        self.b = b

    def midpoint(self):
        """
        The function to find midpoint of this line segment.
        Returns:
            The midpoint of this line segment.
        """
        x = (self.a.x + self.b.x) / 2.0
        y = (self.a.y + self.b.y) / 2.0

        return Point(x, y)

    def intersection(self, line):
        """
        The function to find intersection between this line and the given one.
        Arguments:
            line: The line to test intersection against.
        Returns:
            The tuple with the first value indicating if intersection was found (True/False)
            and the second value holding the intersection Point or None
        """
        A, B, C, D = self.a, self.b, line.a, line.b

        rTop = (A.y - C.y) * (D.x - C.x) - (A.x - C.x) * (D.y - C.y)
        rBot = (B.x - A.x) * (D.y - C.y) - (B.y - A.y) * (D.x - C.x)

        sTop = (A.y - C.y) * (B.x - A.x) - (A.x - C.x) * (B.y - A.y)
        sBot = (B.x - A.x) * (D.y - C.y) - (B.y - A.y) * (D.x - C.x)

        if rBot == 0 or sBot == 0:
            # lines are parallel
            return False, None

        r = rTop / rBot
        s = sTop / sBot
        if r > 0 and r < 1 and s > 0 and s < 1:
            x = A.x + r * (B.x - A.x)
            y = A.y + r * (B.y - A.y)
            return True, Point(x, y)

        return False, None

    def distance(self, p):
        """
        The function to estimate distance to the given point from this line.
        Arguments:
            p: The point to find distance to.
        Returns:
            The distance between given point and this line.
        """
        utop = (p.x - self.a.x) * (self.b.x - self.a.x) + (p.y - self.a.y) * (self.b.y - self.a.y)
        ubot = self.a.distance(self.b)
        ubot *= ubot
        if ubot == 0.0:
            return 0.0

        u = utop / ubot
        if u < 0 or u > 1:
            d1 = self.a.distance(p)
            d2 = self.b.distance(p)
            if d1 < d2:
                return d1
            return d2
        
        x = self.a.x + u * (self.b.x - self.a.x)
        y = self.a.y + u * (self.b.y - self.a.y)
        point = Point(x, y)
        return point.distance(p)

    def length(self):
        """
        The function to calculate the length of this line segment.
        Returns:
            The length of this line segment as distance between its endpoints.
        """
        return self.a.distance(self.b) 

    def __str__(self):
        """
        Returns the nicely formatted string representation of this line.
        """
        return "Line (%.1f, %.1f) -> (%.1f, %.1f)" % (self.a.x, self.a.y, self.b.x, self.b.y)

maze_config.ini

[NEAT]
fitness_criterion     = max
fitness_threshold     = 1.0
pop_size              = 250
reset_on_extinction   = False

[DefaultGenome]
# node activation options
activation_default      = sigmoid
activation_mutate_rate  = 0.0
activation_options      = sigmoid

# node aggregation options
aggregation_default     = sum
aggregation_mutate_rate = 0.0
aggregation_options     = sum

# node bias options
bias_init_mean          = 0.0
bias_init_stdev         = 1.0
bias_max_value          = 30.0
bias_min_value          = -30.0
bias_mutate_power       = 0.5
bias_mutate_rate        = 0.7
bias_replace_rate       = 0.1

# genome compatibility options
compatibility_disjoint_coefficient = 1.1
compatibility_weight_coefficient   = 0.5

# connection add/remove rates
conn_add_prob           = 0.5
conn_delete_prob        = 0.5

# connection enable options
enabled_default         = True
enabled_mutate_rate     = 0.01

feed_forward            = False
initial_connection      = partial_direct 0.5

# node add/remove rates
node_add_prob           = 0.1
node_delete_prob        = 0.1

# network parameters
num_hidden              = 1
num_inputs              = 10
num_outputs             = 2

# node response options
response_init_mean      = 1.0
response_init_stdev     = 0.0
response_max_value      = 30.0
response_min_value      = -30.0
response_mutate_power   = 0.0
response_mutate_rate    = 0.0
response_replace_rate   = 0.0

# connection weight options
weight_init_mean        = 0.0
weight_init_stdev       = 1.0
weight_max_value        = 30
weight_min_value        = -30
weight_mutate_power     = 0.5
weight_mutate_rate      = 0.8
weight_replace_rate     = 0.1

[DefaultSpeciesSet]
compatibility_threshold = 3.0

[DefaultStagnation]
species_fitness_func = max
max_stagnation       = 20
species_elitism      = 1

[DefaultReproduction]
elitism            = 2
survival_threshold = 0.1
min_species_size   = 2

maze_environment.py


import math

import agent
import geometry

# The maximal allowed speend for the maze solver agent
MAX_AGENT_SPEED = 3.0

class MazeEnvironment:
    """
    This class encapsulates the maze simulation environments.
    """
    def __init__(self, agent, walls, exit_point, exit_range=5.0):
        """
        Creates new maze environment with specified walls and exit point.
        Arguments:
            agent: The maze navigating agent
            walls: The maze walls
            exit_point: The maze exit point
            exit_range: The range arround exit point marking exit area
        """
        self.walls = walls
        self.exit_point = exit_point
        self.exit_range = exit_range
        # The maze navigating agent
        self.agent = agent
        # The flag to indicate if exit was found
        self.exit_found = False
        # The initial distance of agent from exit
        self.initial_distance = self.agent_distance_to_exit()

        # Update sendors
        self.update_rangefinder_sensors()
        self.update_radars()
    
    def agent_distance_to_exit(self):
        """
        The function to estimate distance from maze solver agent to 
        the maze exit.
        Returns:
            The distance from maze solver agent to the maze exit.
        """
        return self.agent.location.distance(self.exit_point)
    
    def test_wall_collision(self, loc):
        """
        The function to test if agent at specified location collides
        with any of the maze walls.
        Arguments:
            loc: The new agent location to test for collision.
        Returns:
            The True if agent at new location will collide with any of
            the maze walls.
        """
        for w in self.walls:
            if w.distance(loc) < self.agent.radius:
                return True
        
        return False
    
    def create_net_inputs(self):
        """
        The function to create the ANN input values from the simulation environment.
        Returns:
            The list of ANN inputs consist of values get from solver agent sensors.
        """
        inputs = []
        # the range finders
        for ri in self.agent.range_finders:
            inputs.append(ri)
        
        # The radar sensors
        for rs in self.agent.radar:
            inputs.append(rs)
        
        return inputs

    def apply_control_signals(self, control_signals):
        """
        The function to apply control signals received from control ANN to the
        maze solver agent.
        Arguments:
            control_signals: The contral signals received from the control ANN
        """
        self.agent.angular_vel += (control_signals[0] - 0.5)
        self.agent.speed += (control_signals[1] - 0.5)

        # contrain the speed & angular velocity
        if self.agent.speed > MAX_AGENT_SPEED:
            self.agent.speed = MAX_AGENT_SPEED
        
        if self.agent.speed < -MAX_AGENT_SPEED:
            self.agent.speed = -MAX_AGENT_SPEED

        if self.agent.angular_vel > MAX_AGENT_SPEED:
            self.agent.angular_vel = MAX_AGENT_SPEED
        
        if self.agent.angular_vel < -MAX_AGENT_SPEED:
            self.agent.angular_vel = -MAX_AGENT_SPEED
    
    def update_rangefinder_sensors(self):
        """
        The function to update the agent range finder sensors.
        """
        for i,angle in enumerate(self.agent.range_finder_angles):
            rad = geometry.deg_to_rad(angle)
            # project a point from agent location outwards
            projection_point = geometry.Point(
                x = self.agent.location.x + math.cos(rad) * \
                    self.agent.range_finder_range,
                y = self.agent.location.y + math.sin(rad) * \
                    self.agent.range_finder_range
            )
            # rotate the projection point by the agent's heading 
            # anglet to align it with heading dirction
            projection_point.rotate(
                self.agent.heading,self.agent.location
            )
            # create the line segment from the agent location to
            # the projected point
            projection_line = geometry.Line(
                a = self.agent.location,
                b = projection_point
            )

            # set range to maximum detection range
            min_range = self.agent.range_finder_range

            # new test against maze walls to see if projection line
            # hits any wall and find the closest hit
            for wall in self.walls:
                found, intersection = wall.intersection(projection_line)
                if found:
                    found_range = intersection.distance(self.agent.location)
                    # we are interested in the closest hit
                    if found_range < min_range:
                        min_range = found_range

            # update sensor value
            self.agent.range_finders[i] = min_range

    def update_radars(self):
        """
        The function to update the agent radar sensors.
        """
        target = geometry.Point(self.exit_point.x, self.exit_point.y)
        # rotate target with respect to the agent's heading to align it with 
        # heading direction
        target.rotate(self.agent.heading, self.agent.location)
        # translate with respect to the agent's location
        target.x -= self.agent.location.x
        target.y -= self.agent.location.y
        # the angle between maze eixt point and the agent's heading direction
        angle = target.angle()
        # find the appropriate radar sensor to be fired
        for i, r_angles in enumerate(self.agent.radar_angles):
            self.agent.radar[i] = 0.0 # reset specific radar
            if (angle >= r_angles[0] and angle < r_angles[1]) or \
                (angle + 360 >= r_angles[0] and angle + 360 < r_angles[1]):
                # fire the radar
                self.agent.radar[i] = 1.0
    
    def update(self,control_signals):
        """
        The function to update solver agent position within maze.
        After agent position updated it will be checked to find out if maze
        was reached after that.
        Arguments:
            control_signals: The control signals received from the control ANN
        Returns:
            The True if maze exit was found after update or exit was
            already found in previous simulation cycles.
        """
        if self.exit_found:
            return True # Maze exit already found
        
        # Apply control signals
        self.apply_control_signals(control_signals)

        # get X and Y velocity components
        vx = math.cos(geometry.deg_to_rad(self.agent.heading)) * \
            self.agent.speed
        vy = math.sin(geometry.deg_to_rad(self.agent.heading)) * \
            self.agent.speed
        
        # update current agent's heading (we consider the simulation time
        # step size equal to 1s and the angular velocity as per second)
        self.agent.heading += self.agent.angular_vel

        # Enforce angular velocity bounds by wrapping
        if self.agent.heading > 360:
            self.agent.heading -= 360
        elif self.agent.heading < 0:
            self.agent.heading += 360

        # find the next location of the agent
        new_loc = geometry.Point(
            x = self.agent.location.x + vx,
            y = self.agent.location.y + vy
        )

        if not self.test_wall_collision(new_loc):
            self.agent.location = new_loc
        
        # update agent's sensors
        self.update_rangefinder_sensors()
        self.update_radars()

        # check if agent reached exit point
        distance = self.agent_distance_to_exit()
        self.exit_found = (distance < self.exit_range)
        return self.exit_found
    
    def __str__(self):
        """
        Returns the nicely formatted string representation of this environment.
        """
        string = "MAZE\nAgent at: (%.1f, %.1f)" % (self.agent.location.x,self.agent.location.y)
        string += "\nExit at: (%.1f, %.1f), exit range: %.1f" % (self.exit_point.x,self.exit_found.y,self.exit_range)
        string += "\nWalls [%d]" % len(self.walls)
        for w in self.walls:
            string += "\n\t%s" % w
        
        return string

def read_environment(file_path):
    """
    The function to read maze environment configuration
    from provided file.
    Arguments:
        file_path: The path to the file to read maze configuration from.
    Returns:
        The initialized environment.
    """
    num_lines, index = -1, 0
    walls = []
    maze_agent, maze_exit = None, None
    with open(file_path, 'r') as f:
        for line in f.readlines():
            line = line.strip()
            if len(line) == 0:
                # skip empty lines
                continue
            if index == 0:
                # read the number of line segments
                num_lines = int(line)
            elif index == 1:
                # read the agent's position
                loc = geometry.read_point(line)
                maze_agent = agent.Agent(location=loc)
            elif index == 2:
                # read the agent's initial heading
                maze_agent.heading = float(line)
            elif index == 3:
                # read the maze exit location
                maze_exit = geometry.read_point(line)
            else:
                # read the walls
                wall = geometry.read_line(line)
                walls.append(wall)
            
            # increment cursor
            index += 1
    
    assert len(walls) == num_lines

    print("Maze environment configured successfully from the file: %s" % file_path)
    # create and return the maze environment
    return MazeEnvironment(agent=maze_agent,walls=walls,exit_point=maze_exit)

def maze_simulation_evaluate(env,net,time_steps):
    """
    The function to evaluate maze simulation for specific environment
    and controll ANN provided. The results will be saved into provided
    agent record holder.
    Arguments:
        env: The maze configuration environment.
        net: The maze solver agent's control ANN.
        time_steps: The number of time steps for maze simulation.
    """
    for i in range(time_steps):
        if maze_simulation_step(env, net):
            print("Maze solved in %d steps" % (i + 1))
            return 1.0
    
    # Calculate the fitness score based on distance drom exit
    fitness = env.agent_distance_to_exit()
    # Normalize fitness score to range (0,1]
    fitness = (env.initial_distance - fitness) / env.initial_distance
    if fitness <= 0.01:
        fitness = 0.01
    
    return fitness

def maze_simulation_step(env,net):
    """
    The function to perform one step of maze simulation.
    Arguments:
        env: The maze configuration environment.
        net: The maze solver agent's control ANN
    Returns:
        The True if maze agent solved the maze.
    """
    # create inputs from the current state of the environment
    inputs = env.create_net_inputs()
    # load inputs into controll ANN and get results
    output = net.activate(inputs)
    # apply control signal to the environment and update
    return env.update(output)

maze_experiment.py

import os
import shutil
import math
import random
import time
import copy
import argparse

import neat
import visualize
import utils

import maze_environment as maze
import agent

# The current working directory
local_dir = os.path.dirname(__file__)
# The directory to store outputs
out_dir = os.path.join(local_dir,'out')
out_dir = os.path.join(out_dir,'maze_objective')

class MazeSimulationTrial:
    """
    The class to hold maze simulator execution parameters and results.
    """
    def __init__(self,maze_env,population):
        """
        Creates new instance and initialize fileds.
        Arguments:
            maze_env: The maze environment as loaded from configuration file.
            population: The population for this trial run
        """
        # The initial maze simulation environment
        self.orig_maze_environment = maze_env
        # The record store for evaluated maze solver agents
        self.record_store = agent.AgentRecordStore()
        # The NEAT population object
        self.population = population

trialSim = None

def eval_fitness(genome_id,genome,config,time_steps=400):
    """
    Evaluates fitness of the provided genome.
    Arguments:
        genome_id: The ID of genome.
        genome: The genome to evaluate.
        config: The NEAT configuration holder.
        time_steps: The number of time steps to execute for maze solver
            simulation.
    Returns:
        The phenotype fitness score in range (0,1]
    """
    # run the simulaion
    maze_env = copy.deepcopy(trialSim.orig_maze_environment)
    control_net = neat.nn.FeedForwardNetwork.create(genome, config)
    fitness = maze.maze_simulation_evaluate(
        env=maze_env,net=control_net,time_steps=time_steps
    )

    # Store simulation results into the agent record
    record = agent.AgentRecord(
        generation=trialSim.population.generation,
        agent_id=genome_id
    )
    record.fitness = fitness
    record.x = maze_env.agent.location.x
    record.y = maze_env.agent.location.y
    record.hit_exit = maze_env.exit_found
    record.species_id = trialSim.population.species.get_species_id(genome_id)
    record.species_age = record.generation - \
        trialSim.population.species.get_species(genome_id).created
    # add record to the store
    trialSim.record_store.add_record(record)

    return fitness

def eval_genomes(genomes,config):
    """
    The function to evaluate the fitness of each genome in 
    the genomes list.
    Arguments:
        genomes: The list of genomes from population in the
            current generation
        config: The configuration settings with algorithm
            hyper-parameters
    """
    for genome_id,genome in genomes:
        genome.fitness = eval_fitness(genome_id, genome, config)

def run_experiment(
    config_file,maze_env,trial_out_dir,args=None,n_generations=100,silent=False):
    """
    The function to run the experiment against hyper-parameters defined in the 
    provided configuration file.
    The winner genome will be rendered as a graph as well as the import statictics
    of neuroevolution process execution.
    Arguments:
        config_file: The path to the file with experiment configuration
        maze_env: The maze environment to use in simulation.
        trial_out_dir: The directory to store outputs for this trial
        n_generations: The number of generations to execute.
        silent: If True than no intermediary outputs will be presented
            until solution is found.
        args: The command line argument holder.
    """
    # seed = 1559231616
    # random.seed(seed)

    # Load configuration
    config = neat.Config(neat.DefaultGenome,neat.DefaultReproduction,
        neat.DefaultSpeciesSet,neat.DefaultStagnation,config_file)

    # Create the population, which is the top-level object for a NEAT run.
    p = neat.Population(config)

    # create the trial simulation
    global trialSim
    trialSim = MazeSimulationTrial(maze_env=maze_env, population=p)

    # Add a stdout reporter to show progress in the terminal.
    p.add_reporter(neat.StdOutReporter(True))
    stats = neat.StatisticsReporter()
    p.add_reporter(stats)
    p.add_reporter(neat.Checkpointer(
        5,filename_prefix='%s/maze-neat-checkpoint-' % trial_out_dir))

    # Run for up to N generations.
    start_time = time.time()
    best_genome = p.run(eval_genomes, n=n_generations)

    elapsed_time = time.time() - start_time

    # Display the best genome among generations.
    print('\nBest genome:\n%s' % (best_genome))

    solution_found = (best_genome.fitness >= config.fitness_threshold)
    if solution_found:
        print("SUCCESS: The stable maze solver controller was found!!!")
    else:
        print("FAILURE: Failed to find the stable maze solver controller!!!")
    
    # write the record store data
    rs_file = os.path.join(trial_out_dir,'data.pickle')
    trialSim.record_store.dump(rs_file)

    print("Record store file: %s" % rs_file)
    #print("Random seed:",seed)
    print("Trial elapsed time: %.3f sec" % (elapsed_time))

    # Visualize the experiment results
    if not silent or solution_found:
        node_names = {
            -1:'RF_R',-2:'RF_FR',-3:'RF_F',-4:'RF_FL',
            -5:'RF_F',-6:'RF_B',-7:'RAD_F',-8:'RAD_F',
            -9:'RAD_B',-10:'RAD_R',0:'ANG_VEL',1:'VEL'
        }
        visualize.draw_net(config,best_genome,True,node_names=node_names,directory=trial_out_dir,fmt='svg')
        if args is None:
            visualize.draw_maze_records(maze_env,trialSim.record_store.records,view=True)
        else:
            visualize.draw_maze_records(
                maze_env,trialSim.record_store.records,
                view=True,width=args.width,height=args.height,
                filename=os.path.join(trial_out_dir,'maze_records.svg')
            )
            visualize.plot_stats(
                stats,ylog=False,view=True,
                filename=os.path.join(trial_out_dir,'avg_fitness.svg')
            )
            visualize.plot_species(
                stats,view=True,
                filename=os.path.join(trial_out_dir,'speciation.svg')
            )
    
    return solution_found

if __name__ == '__main__':
    # read command line parameters
    parser = argparse.ArgumentParser(description="The maze experiment runner.")
    parser.add_argument('-m','--maze',default='medium',help='The maze configuration to use.')
    parser.add_argument('-g','--generations',default=500,type=int,help='The number of generations for the evolutionary process')
    parser.add_argument('--width',type=int,default=400,help='The width of the records subplot')
    parser.add_argument('--height',type=int,default=400,help='The height of the records subplot')
    args = parser.parse_args()
    
    if not (args.maze == 'medium' or args.maze == 'hard'):
        print("Unsupported maze configuration: %s" % args.maze)
        exit(1)
    
    # Determine path to configuration file.
    config_path = os.path.join(local_dir,'maze_config.ini')

    trial_out_dir = os.path.join(out_dir,args.maze)

    # Clean results of previous run if any or init the output directory
    utils.clear_output(trial_out_dir)

    # run the experiment
    maze_env_config = os.path.join(local_dir, '%s_maze.txt' % args.maze)
    maze_env = maze.read_environment(maze_env_config)

    # visualize.draw_maze_records(maze_env,None,view=True)

    print("Startting the %s maze experiment" % args.maze)
    run_experiment(
        config_file=config_path,maze_env=maze_env,
        trial_out_dir=trial_out_dir,
        n_generations=args.generations,
        args=args
    )

medium_maze.txt

11
30 22
0
270 100
5 5 295 5
295 5 295 135
295 135 5 135
5 135 5 5
241 135 58 65
114 5 73 42
130 91 107 46
196 5 139 51
219 125 182 63
267 5 214 63
271 135 237 88

utils.py

import os
import shutil

def clear_output(out_dir):
    """
    Function to clear output directory.
    Arguments:
        out_dir: The directory to be clear
    """
    if os.path.isdir(out_dir):
        # remove files from previous run
        shutil.rmtree(out_dir)
    
    # create the output directory
    os.makedirs(out_dir,exist_ok=False)

visualize.py

from __future__ import print_function

import copy
import warnings
import random
import argparse
import os

import graphviz
import matplotlib.pyplot as plt
import matplotlib.lines as mlines
import matplotlib.patches as mpatches
import numpy as np

import geometry
import agent
import maze_environment as maze

def plot_stats(statistics,ylog=False,view=False,filename='avg_fitness.svg'):
    """
    Plots the population's average and best fitness.
    """
    if plt is None:
        warnings.warn("This display is not available due to a missing optional dependency (matplotlib)")
        return
    
    generation = range(len(statistics.most_fit_genomes))
    best_fitness = [c.fitness for c in statistics.most_fit_genomes]
    avg_fitness = np.array(statistics.get_fitness_mean())
    stdev_fitness = np.array(statistics.get_fitness_stdev())

    plt.plot(generation, avg_fitness, 'b-', label='average')
    plt.plot(generation, avg_fitness - stdev_fitness, 'g-.', label='-1 sd')
    plt.plot(generation, avg_fitness + stdev_fitness, 'g-', label='+1 sd')
    plt.plot(generation, best_fitness, 'r-', label='best')

    plt.title("Population's average and best fitness")
    plt.xlabel("Generations")
    plt.ylabel("Fitness")
    plt.grid()
    plt.legend(loc="best")
    if ylog:
        plt.gca().set_yscale('symlog')

    plt.savefig(filename)
    if view:
        plt.show()
    
    plt.close()

def plot_species(statistics,view=False,filename='speciation.svg'):
    """
    Visualizes speciation throughout evolution.
    """
    if plt is None:
        warnings.warn("This display is not available due to a missing optional dependency (matplotlib)")
        return
    
    species_sizes = statistics.get_species_sizes()
    num_generations = len(species_sizes)
    curves = np.array(species_sizes).T

    fig, ax = plt.subplots()
    ax.stackplot(range(num_generations), *curves)

    plt.title("Speciation")
    plt.ylabel("Size per Species")
    plt.xlabel("Generations")

    plt.savefig(filename)
    
    if view:
        plt.show()

    plt.close()

def draw_net(
    config,genome,view=False,filename=None,directory=None,
    node_names=None,show_disabled=True,prune_unused=False,
    node_colors=None,fmt='svg'):
    """
    Recevies a genome and draws a neural network with arbitrary topolotgy.
    """
    # Attributes for network nodes
    if graphviz is None:
        warnings.warn("This display is not available due to a missing optional dependency (graphviz)")
        return
    
    if node_names is None:
        node_names = {}
    
    assert type(node_names) is dict

    if node_colors is None:
        node_colors = {}
    
    assert type(node_colors) is dict

    node_attrs = {
        'shape': 'circle',
        'fontsize': '9',
        'height': '0.2',
        'width': '0.2'
    }

    dot = graphviz.Digraph(format=fmt, node_attr=node_attrs)

    inputs = set()
    for k in config.genome_config.input_keys:
        inputs.add(k)
        name = node_names.get(k, str(k))
        input_attrs = {
            'style':'filled',
            'shape':'box',
            'fillcolor':node_colors.get(k,'lightgray')
        }

        dot.node(name, _attributes=input_attrs)
    
    outputs = set()
    for k in config.genome_config.output_keys:
        outputs.add(k)
        name = node_names.get(k, str(k))
        node_attrs = {
            'style':'filled',
            'fillcolor':node_colors.get(k,'lightblue')
        }

        dot.node(name, _attributes=node_attrs)
    
    if prune_unused:
        connections = set()
        for cg in genome.connections.values():
            if cg.enabled or show_disabled:
                connections.add((cg.in_node_id, cg.out_node_id))
        
        used_nodes = copy.copy(outputs)
        pending = copy.copy(outputs)
        while pending:
            new_pending = set()
            for a, b in connections:
                if b in pending and a not in used_nodes:
                    new_pending.add(a)
                    used_nodes.add(a)
            pending = new_pending
    else:
        used_nodes = set(genome.nodes.keys())
    
    for n in used_nodes:
        if n in inputs or n in outputs:
            continue

        attrs = {
            'style':'filled',
            'fillcolor':node_colors.get(n,'white')
        }
        dot.node(str(n), _attributes=attrs)

    for cg in genome.connections.values():
        if cg.enabled or show_disabled:
            # if cg.input not in used_nodes or cg.output not in used_nodes:
            #     continue
            input_node,output_node = cg.key
            a = node_names.get(input_node,str(input_node))
            b = node_names.get(output_node,str(output_node))
            style = 'solid' if cg.enabled else 'dotted'
            color = 'green' if cg.weight > 0 else 'red'
            width = str(0.1 + abs(cg.weight / 5.0))
            dot.edge(a,b,_attributes={
                'style':style,'color':color,'penwidth':width
            })
    
    dot.render(filename, directory, view=view)

    return dot

def draw_maze_records(
    maze_env,records,best_threshold=0.8,filename=None,
    view=False,show_axes=False,width=400,height=400,fig_height=7):
    """
    The function to draw maze with recorded agents positions
    Arguments:
        maze_env: The maze environment configuration
        records: The records of solver agents collected during NEAT execution.
        best_threshold: The minimal fitness of maze solving agent's species to
            be included into the best ones
        view: The flag to indicate whether to view plot.
        width: The width of drawing in pixels
        height: The height of drawing in pixels
        fig_height: The plot figure height in inches
    """
    # find the distance threshold for the best species
    dist_threshold = maze_env.agent_distance_to_exit() * (1.0 - best_threshold)
    # generate color palette and find the best species IDS
    max_sid = 0
    for r in records:
        if r.species_id > max_sid:
            max_sid = r.species_id
    colors = [None] * (max_sid + 1)
    sp_idx = [False] * (max_sid + 1)
    best_sp_idx = [0] * (max_sid + 1)
    for r in records:
        if not sp_idx[r.species_id]:
            sp_idx[r.species_id] = True
            rgb = (random.random(), random.random(), random.random())
            colors[r.species_id] = rgb
        if maze_env.exit_point.distance(geometry.Point(r.x, r.y)) <= dist_threshold:
            best_sp_idx[r.species_id] += 1

    # initialize plotting
    fig = plt.figure()
    fig.set_dpi(100)
    fig_width = fig_height * (float(width) / float(2.0 * height)) - 0.2
    print("Plot figure width: %.1f, height: %.1f" % (fig_width,fig_height))
    fig.set_size_inches(fig_width,fig_height)
    ax1,ax2 = fig.subplots(2,1,sharex=True)
    ax1.set_xlim(0, width)
    ax1.set_ylim(0, height)
    ax2.set_xlim(0, width)
    ax2.set_ylim(0, height)

    # draw species
    n_best_species = 0
    for i,v in enumerate(best_sp_idx):
        if v > 0:
            n_best_species += 1
            _draw_species_(records=records, sid=i, colors=colors, ax=ax1)
        else:
            _draw_species_(records=records, sid=i, colors=colors, ax=ax2)
    
    ax1.set_title('fitness >= %.1f, species: %d' % (best_threshold, n_best_species))
    ax2.set_title('fitness < %.1f' % best_threshold)

    # draw maze
    _draw_maze_(maze_env, ax1)
    _draw_maze_(maze_env, ax2)

    if not show_axes:
        ax1.axis('off')
        ax2.axis('off')
    
    # Invert Y axis to have coordinates origin at the top left
    ax1.invert_yaxis()
    ax2.invert_yaxis()

    # save figure to file
    if filename is not None:
        plt.savefig(filename)
    
    if view:
        plt.show()
    
    plt.close()

def _draw_species_(records,sid,colors,ax):
    """
    The function to draw specific species from the records
    with particular color.
    Arguments:
        records: The records of solver agents collected during NEAT execution
        sid: The species ID
        colors: The colors table by species ID
        ax: The figure axis instance
    """
    for r in records:
        if r.species_id == sid:
            circle = plt.Circle((r.x, r.y), 2.0, facecolor=colors[r.species_id])
            ax.add_patch(circle)

def _draw_maze_(maze_env,ax):
    """
    The function to draw maze environment
    Arguments:
        maze_env: The maze environment configuration.
        ax: The figure axis instance
    """
    # draw maze walls
    for wall in maze_env.walls:
        line = plt.Line2D((wall.a.x, wall.b.x), (wall.a.y, wall.b.y), lw=1.5)
        ax.add_line(line)
    
    # draw start point
    start_circle = plt.Circle(
        (maze_env.agent.location.x, maze_env.agent.location.y),
        radius = 2.5, facecolor = (0.6, 1.0, 0.6),edgecolor='w',
    )
    ax.add_patch(start_circle)

    # draw exit point
    exit_circle = plt.Circle(
        (maze_env.exit_point.x,maze_env.exit_point.y),
        radius = 2.5, facecolor = (1.0,0.2,0.0),edgecolor='w'
    )
    ax.add_patch(exit_circle)

if __name__ == '__main__':
    # read command line parameters
    parser = argparse.ArgumentParser(description='The maze experiment visualizer.')
    parser.add_argument('-m', '--maze', default='medium', help='The maze configuration to use.')
    parser.add_argument('-r', '--records', help='The records files.')
    parser.add_argument('-o', '--output', help='The file to store the plot.')
    parser.add_argument('--width', type=int, default=400, help='The width of the subplot')
    parser.add_argument('--height',type=int,default=400,help='The height of the subplot')
    parser.add_argument('--fig_height',type=float,default=7,help='The height of the plot figure')
    parser.add_argument('--show_axes',type=bool,default=False,help='The flag to indicate whether to show plot axes.')
    args = parser.parse_args()

    local_dir = os.path.dirname(__file__)
    if not (args.maze == 'medium' or args.maze == 'hard'):
        print("unsupported maze configuration: %s" % args.maze)
        exit(1)

    # read maze environment
    maze_env_config = os.path.join(local_dir,'%s_maze.txt' % args.maze)
    maze_env = maze.read_environment(maze_env_config)

    # read agents records
    rs = agent.AgentRecordStore()
    rs.load(args.records)

    # render visualization
    random.seed(42)
    draw_maze_records(
        maze_env,rs.records,width=args.width,
        height=args.height,fig_height=args.fig_height,
        view=True,show_axes=args.show_axes,
        filename=args.output
    )

盼小辉丶

关注

5
点赞
踩
8

收藏

觉得还不错? 一键收藏
打赏
3
评论
神经进化算法——利用NEAT算法解决迷宫导航问题（基于NEAT-Python）

迷宫导航问题是经典的计算机问题，与创建可以在模糊环境中找到路径的自主导航智能体密切相关。迷宫环境中，面向目标的适应度函数在迷宫中接近最终目标点的死角中可以具有较大的适应度得分梯度。迷宫的这些区域成为基于目标搜索算法的局部最优解，这些目标算法可能会收敛于这些区域。利用神经进化算法NEAT来解决迷宫导航问题。基于NEAT-python利用NEAT算法得到最终的人工神经网络来控制智能体的速度和角速度，最终到达迷宫的出口点。
复制链接

扫一扫