self.fitness = fitness
评估的基因组的新颖性评分
self.novelty = novelty
self.in_archive = False
数据点的列表，这些数据点表示迷宫求解器智能体在模拟过程中访问的特定迷宫位置的坐标。
该数据列表用于估计当前新奇项和其他新奇项之间的距离。
self.data = []
NoveltyArchive
该类维护了一系列相关的新奇项，并提供了评估单个基因组以及整个基因组总体的新颖性评分的方法。它在构造函数中定义了以下字段：
def init(self, threshold, metric):
用于估计新颖性度量的函数引用
self.novelty_metric = metric
添加到此存档中的NoveltyItem的当前最小新奇得分
self.novelty_threshold = threshold
novelty_threshold的最小可能值
self.novelty_floor = 0.25
用于novelty_threshold值的动态更改
self.items_added_in_generation = 0
self.time_out = 0
k个最近邻居的默认数量，可用于新奇评分估计
self.neighbors = KNNNoveltyScore
当前的进化世代
self.generation = 0
当前收集的所有相关NoveltyItem实例的列表
self.novel_items = []
所有对象中具有最大的面向目标适应性得分的新颖项目的列表
self.fittest_items = []
novelty_threshold字段的动态变化取决于以下源代码：
def _adjust_archive_settings(self):
“”"
The function to adjust the dynamic novelty threshold depending
on how many have NoveltyItem objects have been added to the archive recently
“”"
if self.items_added_in_generation == 0:
self.time_out += 1
else:
self.time_out = 0
如果在10代中未找到新的路径，则将novelty_threshold值降低5％
if self.time_out >= 10:
self.novelty_threshold *= 0.95
if self.novelty_threshold < self.novelty_floor:
self.novelty_threshold = self.novelty_floor
self.time_out = 0
如果在上一代中添加了四个以上的新奇项，则将novelty_threshold值提高20％
if self.items_added_in_generation >= 4:
self.novelty_threshold *= 1.2
reset counters
self.items_added_in_generation = 0
在每代进化完成后调用此函数，以调整下一代的novelty_threshold字段值。该值确定了下一代应向archive中添加多少个新颖性项，以适应随着时间的推移使用NS方法查找新颖解的难度。
以下源代码显示了如何使用novelty_threshold值来确定要添加哪个NoveltyItem：
def evaluate_individual_novelty(self, genome, genomes, n_items_map, only_fitness=False):
if genome.key not in n_items_map:
print(“WARNING! Found Genome without novelty point associated: %s” +
“\nNovelty evaluation will be skipped for it. Probably winner found!” % genome.key)
return
item = n_items_map[genome.key]
if item.fitness == -1.0:
return -1.0
result = 0.0
if only_fitness:
result = self._novelty_avg_knn(item=item, genomes=genomes, n_items_map=n_items_map)
else:
result = self._novelty_avg_knn(item=item, neighbors=1, n_items_map=n_items_map)
if result > self.novelty_threshold or len(self.novel_items) < ArchiveSeedAmount:
self._add_novelty_item(item)
item.novelty = result
item.generation = self.generation
return result
前面的代码使用函数来评估新颖性得分，以估计所提供基因组的新颖性。如果在更新存档模式下调用此函数(only_fitness = False)，则将获得的新奇得分（result）与novelty_threshold字段的当前值进行比较。根据比较结果，是否将NoveltyItem对象添加到NoveltyArchive对象。此外，引入了ArchiveSeedAmount常量，以在存档仍然为空时在演化开始时使用NoveltyItem实例对存档进行初始化。
新颖性得分
智能体的行为空间由运行迷宫求解模拟时通过迷宫的轨迹定义。因此，任何行为空间访问点密集的区域都不那么新颖，对求解器代理程序的奖励也更少。
最简单的衡量点稀疏度的方法是从它到k个近邻的平均距离。稀疏区域具有较高的距离值，而较稠密的区域具有较低的距离值。以下公式给出了行为空间点的稀疏性：
ρ = 1 k ∑ i = 0 k d i s t ( x , u i ) \rho = \frac 1k\sum_{i=0}^kdist(x,u_i) ρ=k1i=0∑kdist(x,ui)
其中， u i u_i ui是距离（新颖性） d i s t ( x , y ) dist(x,y) dist(x,y)度量计算的第i个最接近的邻居。
通过上述公式在行为空间中特定点的稀疏度计算出的新颖性分数可以由适应度函数使用。
以下函数定义了用于查找新颖性分数的Python代码：
def _novelty_avg_knn(self, item, n_items_map, genomes=None, neighbors=None):
distances = None
判断是否包含当前种群中所有基因组的列表，若包含，则首先填充种群中所有基因组的行为特征之间的距离列表，包括来自NoveltyArchive的所有NoveltyItem对象。否则，将使用提供的新颖项（item）从NoveltyArchive查找其与所有NoveltyItem对象之间的距离
if genomes is not None:
distances = self._map_novelty_in_population(item=item, genomes=genomes, n_items_map=n_items_map)
else:
distances = self._map_novelty(item=item)
按照从小到大的顺序对距离列表进行排序
distances.sort()
初始化计算k最近邻居分数所需的所有中间变量，并测试上一步中收集的距离值的数量是否大于ArchiveSeedAmount常数值
if neighbors is None:
neighbors = self.neighbors
density, weight, distance_sum = 0.0, 0.0, 0.0
length = len(distances)
检查找到的距离列表的长度是否小于针对其进行测试的邻居的数量（neighbors）
if length >= ArchiveSeedAmount:
length = neighbors
if len(distances) < length:
the number of mapped distances is less than number of neighbors
length = len(distances)
循环获取所有距离和权重之和
i = 0
while weight < float(neighbors) and i < length:
distance_sum += distances[i].distance
weight += 1.0
i += 1
由于计算出的权重值超过了指定的邻居数而导致前一个循环退出时，或者如果已经对distances列表中的所有距离值进行了迭代，则可以将给定项目的新颖性得分计算为平均距离到k个最近的邻居
if weight > 0:
density = distance_sum / weight
return density
新颖性指标
新颖性指标是衡量当前解决方案与已知解决方案有多不同的一种度量。当估计从行为空间中的当前点到它的k个最近邻居的距离时，它用于计算新奇分数。
在实验中，通过两个轨迹向量（每个智能体一个向量）之间的逐项距离来确定测量行为差异的新颖性度量，轨迹矢量包含迷宫导航代理在仿真过程中访问的位置的坐标：
d i s t ( x , μ ) = 1 n ∑ j = 0 n ∣ x j − u j ∣ dist(x,\mu)=\frac 1n\sum_{j=0}^n|x_j-u_j| dist(x,μ)=n1j=0∑n∣xj−uj∣
其中n是轨迹向量 μ j \mu_j μj和 x j x_j xj的大小。
在迷宫导航实验中，主要对智能体的最终位置感兴趣。因此，最终，轨迹矢量可能仅包含智能体的最终坐标。
用于新颖性度量值估计的Python代码如下：
def maze_novelty_metric(first_item, second_item):
diff_accum = 0.0
size = len(first_item.data)
for i in range(size):
diff = abs(first_item.data[i] - second_item.data[i])
diff_accum += diff
return diff_accum / float(size)
适应度函数
对于不同任务，使用各种适合度因子：
-
新颖性分数用于指导神经进化过程。它被指定为每个基因组的适应度值，并在进化世代中用于基因组评估。
-
从迷宫模拟器获得的面向目标的适应度得分，用于测试是否已实现最终目标（即，已找到迷宫出口），记录此值以高效评估每个智能体。
适应度值评估的源代码通过两个函数提供：
-
回调函数，用于评估整个种群的适应性得分（eval_genomes）
-
通过迷宫求解过程评估单个基因组的函数（eval_individual）
种群适应度评价函数
适应度评估函数是在NEAT-Python库中注册的回调函数，允许该库针对需要解决的特定任务的特定条件对种群基因组进行评估。
NEAT-Python库不允许通过回调函数传递有关任务完成的任何信号，除非通过指定获胜者基因组的特定适应性得分值。该适应性值必须高于NEAT-Python超参数配置中的适应性阈值。但是，使用NS算法，不可能准确估计获胜者基因组可以达到的新奇分数的上限。此外，优胜者基因组的新颖性得分值可以低于在进化过程中较早时基因组获得的值。
因此，鉴于将新奇评分指定为基因组的适应度值，需要一种解决方法，以使用由NEAT-Python库定义的标准终止条件。通过使用一个特定的评分值来做到这一点，此值确定通过NEAT-Python超参数配置提供的终止条件。使用800000作为新奇得分的指示性度量，并将其自然对数（约13.59）作为适应度阈值。
该函数的完整源代码如下：
def eval_genomes(genomes, config):
创建字典以存储种群中每个基因组的评估新颖性项（n_items_map）
n_items_map = {}
solver_genome = None
遍历种群中的所有基因组，评估其迷宫求解性能
for genome_id, genome in genomes:
found = eval_individual(genome_id=genome_id,genome=genome,
genomes=genomes,n_items_map=n_items_map,config=config)
if found:
solver_genome = genome
trial_sim.archive.end_of_generation()
遍历种群中的所有基因组，以使用估计的新颖性分数为基因组分配适应性分数。
在迷宫求解模拟过程中，新颖性分数估算过程使用n_items_map中收集的NoveltyItem对象：
for genome_id, genome in genomes:
fitness = trial_sim.archive.evaluate_individual_novelty(genome=genome,
genomes=genomes,n_items_map=n_items_map,only_fitness=True)
if fitness > 1:
fitness = math.log(fitness)
else:
fitness = 0
genome.fitness = fitness
如果找到了成功的求解器基因组，为其分配的适应度值等于之前所述的指示性适应度得分
if solver_genome is not None:
solver_genome.fitness = math.log(800000) # ~=13.59
个体适应度评价函数
此函数是种群适应性评估的重要组成部分，可以从前面讨论的eval_genomes函数中调用该功能，以评估种群中每个基因组的迷宫解决性能。
通过迷宫导航模拟对单个基因组作为迷宫导航器的评估如下：
def eval_individual(genome_id, genome, genomes, n_items_map, config):
创建NoveltyItem对象，以保存有关与特定基因组相关的新颖性评分的信息，
并将其保存在n_items_map字典的全基因组ID下
n_item = archive.NoveltyItem(generation=trial_sim.population.generation,
genomeId=genome_id)
n_items_map[genome_id] = n_item
创建原始迷宫环境的副本，以避免在仿真过程中产生副作用，然后从提供的基因组创建ANN
maze_env = copy.deepcopy(trial_sim.orig_maze_environment)
control_net = neat.nn.FeedForwardNetwork.create(genome, config)
使用迷宫环境和ANN的副本，针对给定数量的模拟步骤执行迷宫求解模拟
goal_fitness = maze.maze_simulation_evaluate(
env=maze_env,
net=control_net,
time_steps=SOLVER_TIME_STEPS,
n_item=n_item,
mcns=MCNS)
if goal_fitness == -1:
The individual doesn’t meet the minimal fitness criterion
print(“Individ with ID: %d marked for extiction, MCNS: %f”
% (genome_id, MCNS))
return False
返回的基于目标的适应度评分以及其他基因组参数存储在AgentRecord中，然后将其添加到记录存储中
record = agent.AgentRecord(
generation=trial_sim.population.generation,
agent_id=genome_id)
record.fitness = goal_fitness
record.x = maze_env.agent.location.x
record.y = maze_env.agent.location.y
record.hit_exit = maze_env.exit_found
record.species_id = trial_sim.population.species.get_species_id(genome_id)
record.species_age = record.generation - \
trial_sim.population.species.get_species(genome_id).created
add record to the store
trial_sim.record_store.add_record(record)
如果给定的基因组不是最终结果，估计其新奇分数，
用当前基因组的NoveltyItem更新NoveltyArchive中高适应度值的基因组列表：
if not maze_env.exit_found:
evaluate genome novelty and add it to the archive if appropriate
record.novelty = trial_sim.archive.evaluate_individual_novelty(
genome=genome, genomes=genomes,n_items_map=n_items_map)
update fittest organisms list
trial_sim.archive.update_fittest_with_genome(genome=genome,
n_items_map=n_items_map)
return maze_env.exit_found
实验中，基因组的适应度评分定义为两个单独的值，每个值具有不同的用途。面向目标的适应性评分有助于测试是否已找到解，并收集统计信息。基于新颖性的适应度评分指导神经进化过程。
迷宫环境与迷宫求解器智能体介绍
该部分详细介绍参考链接。
超参数选择
实验中使用的目标函数基于没有明确上限值的新颖性度量。因此，不能精确地估计适应度阈值，为了表明找到了胜出的解决方案，使用一个指示值。
选择800000作为指示性新颖性得分。但是，为了在绘制实验结果时保持适应度得分的直观呈现，使用自然对数缩小了求解器的新颖性得分。因此，配置文件中使用的适应性阈值变为13.5，这比最大可能适应度分数（13.59）小一点，以避免出现舍入浮点数的问题：
[NEAT]
fitness_criterion = max
fitness_threshold = 13.5
pop_size = 500
reset_on_extinction = False
物种在进化停滞时的的生存时间更长：
[DefaultStagnation]
max_stagnation = 100
实验运行函数
config = neat.Config(
neat.DefaultGenome,neat.DefaultReproduction,
neat.DefaultSpeciesSet,neat.DefaultStagnation,
config_file)
p = neat.Population(config)
- 为了在每次评估之后保留中间结果，使用MazeSimulationTrial对象初始化trial_sim全局变量。
global trial_sim
trial_sim = MazeSimulationTrial(maze_env=maze_env,population=p,archive=novelty_archive)
p.add_reporter(neat.StdOutReporter(True))
stats = neat.StatisticsReporter()
p.add_reporter(stats)
start_time = time.time()
best_genome = p.run(eval_genomes,n=n_generations)
elapsed_time = time.time() - start_time
Display the best genome among generations.
print(‘\nBest genome: \n%s’ % (best_genome))
solution_found = (best_genome.fitness >= config.fitness_threshold)
if solution_found:
print(“SUCCESS: The stable maze solver controller was found!!!”)
else:
print(“FAILURE: Failed to find the stable maze solver controller!!!”)
node_names = {
-1:‘RF_R’,-2:‘RF_RF’,-3:‘RF_F’,-4:‘RF_FL’,
-5:‘RF_L’,-6:‘RF_B’,-7:‘RAD_F’,-8:‘RAD_L’,
-9:‘RAD_B’,-10:‘RAD_R’,0:‘ANG_VEL’,1:‘VEL’
}
visualize.draw_net(
config,best_genome,view=show_results,
node_names=node_names,directory=trial_out_dir,
fmt=‘svg’)
if args is None:
visualize.draw_maze_records(
maze_env,trial_sim.record_store.records,
view=show_results)
else:
visualize.draw_maze_records(
maze_env,trial_sim.record_store.records,
view=show_results,width=args.width,
height=args.height,
filename=os.path.join(trial_out_dir,‘maze_records.svg’))
visualize.plot_stats(
stats,ylog=False,
filename=os.path.join(trial_out_dir,‘avg_fitness.svg’))
visualize.plot_species(
stats,view=show_results,
filename=os.path.join(trial_out_dir,‘speciation.svg’))
store NoveltyItems archive data
trial_sim.archive.write_fittest_to_file(
path=os.path.join(trial_out_dir,‘ns_items_fittest.txt’))
trial_sim.archive.write_to_file(
path=os.path.join(trial_out_dir,‘ns_items_all.txt’)
)
- 最后，执行其他可视化例程，以可视化迷宫求解器智能体通过迷宫的路径。
maze_env = copy.deepcopy(trial_sim.orig_maze_environment)
control_net = neat.nn.FeedForwardNetwork.create(best_genome,config)
path_points = []
evaluate_fitness = maze.maze_simulation_evaluate(
env=maze_env,net=control_net,
time_steps=SOLVER_TIME_STEPS,
path_points=path_points)
print(“Evaluated fitness of best agent: %f” % evaluate_fitness)
visualize.draw_agent_path(
trial_sim.orig_maze_environment,
path_points,best_genome,
view=show_results,width=args.width,
height=args.height,
filename=os.path.join(trial_out_dir,‘best_solver_path.svg’))
运行实验
在终端中执行以下命令：
$ python3 maze_experiment.py -g 500 -t 10 -m medium --width 300 --height 150
显示获胜者基因组的配置和有关试验的一般统计数据：
Best genome:
Key: 36170
Fitness: 13.592367006650065
Nodes:
0 DefaultNodeGene(key=0, bias=0.9583749969785536, response=1.0, activation=sigmoid, aggregation=sum)
1 DefaultNodeGene(key=1, bias=-1.3352111211865185, response=1.0, activation=sigmoid, aggregation=sum)
Connections:
DefaultConnectionGene(key=(-10, 0), weight=2.012697148953962, enabled=True)
DefaultConnectionGene(key=(-10, 1), weight=2.3586743900645715, enabled=True)
DefaultConnectionGene(key=(-9, 0), weight=0.5133819837545476, enabled=False)
DefaultConnectionGene(key=(-9, 1), weight=-1.3453064468779043, enabled=True)
DefaultConnectionGene(key=(-8, 0), weight=-1.3151248904230235, enabled=True)
DefaultConnectionGene(key=(-6, 1), weight=-1.50551995321142, enabled=True)
DefaultConnectionGene(key=(-5, 0), weight=-3.020445866909174, enabled=False)
DefaultConnectionGene(key=(-5, 1), weight=-2.090540743662507, enabled=True)
DefaultConnectionGene(key=(-4, 0), weight=-1.8754146567384993, enabled=True)
DefaultConnectionGene(key=(-4, 1), weight=2.0773106904549614, enabled=True)
DefaultConnectionGene(key=(-3, 0), weight=2.6406887829938044, enabled=True)
DefaultConnectionGene(key=(-3, 1), weight=0.4049529471735065, enabled=True)
DefaultConnectionGene(key=(-2, 1), weight=0.5571713919237005, enabled=True)
DefaultConnectionGene(key=(-1, 0), weight=1.5212036155782374, enabled=True)
DefaultConnectionGene(key=(-1, 1), weight=0.7204766260373855, enabled=True)
DefaultConnectionGene(key=(0, 0), weight=1.1105019563826226, enabled=True)
SUCCESS: The stable maze solver controller was found!!!
控制器ANN的最终配置如下所示：
进化过程适应度得分曲线图：
进化过程中物种图：
智能体记录可视化
智能体记录的可视化：
$ python3 visualize.py -m medium -r out/maze_ns/medium/0/data.pickle --width 300 --height 150
查看成功的智能体的路径，该智能体能够找到迷宫出口：
- geometry.py、medium_maze.txt、utils.py
代码链接。
- agent.py
import pickle
class Agent:
“”"
This is maze navigating agent.
“”"
def init(
self,location,heading=0,speed=0,angular_vel=0,radius=8.0,
range_finder_range=100.0):
“”"
Creates new Agent with spcified parameters.
Arguments:
location: The agent initial position within maze.
heading: The heading direction in degrees.
speed: The linear velocity of the agent.
angular_vel: The angular velocity of the agent.
radius: The agent’s body radius.
range_finder_range: The maximal detection range for radar sensor.
“”"
self.heading = heading
self.speed = speed
self.angular_vel = angular_vel
self.radius = radius
self.range_finder_range = range_finder_range
self.location = location
defining the range finder sensors
self.range_finder_angles = [-90.0, -45.0, 0.0, 45.0, 90.0, -180.0]
defining the range radar sensors
self.radar_angles = [(315.0, 405.0), (45.0, 135.0), (135.0, 225.0), (225.0, 315.0)]
the list to hold range finders activations
self.range_finders = [None] * len(self.range_finder_angles)
the list to hold pie-slice radar activations
self.radar = [None] * len(self.radar_angles)
class AgentRecord:
“”"
The class to hold results of maze navigation simulation for specific
solver agent. It provides all statistics about the agent at the end
of navigation run.
“”"
def init(self,generation,agent_id):
“”"
Creates new record for specific agent at the specific generation
of the evolutionary process.
“”"
self.generation = generation
self.agent_id = agent_id
initialize agent’s properties
self.x = -1
self.y = -1
self.fitness = -1
self.novelty = -1
The flag to indicate whether this agent was able to find maze exit
self.hit_exit = False
The ID of species this agent belong to
self.species_id = -1
The age of agent’s species at the time of recording
self.species_age = -1
class AgentRecordStore:
“”"
The class to control agents record store.
“”"
def init(self):
“”"
Creates new instance.
“”"
self.records = []
def add_record(self,record):
“”"
The function to add specified record to this store.
Arguments:
record: The record to be added.
“”"
self.records.append(record)
def load(self, file):
“”"
The function to load records list from the specied file into this class.
Arguments:
file: The path to the file to read agents records from.
“”"
with open(file, ‘rb’) as dump_file:
self.records = pickle.load(dump_file)
def dump(self, file):
“”"
The function to dump records list to the specified file from this class.
Arguments:
file: The path to the file to hold data dump.
“”"
with open(file, ‘wb’) as dump_file:
pickle.dump(self.records, dump_file)
- maze_config.ini
[NEAT]
fitness_criterion = max
fitness_threshold = 13.5
pop_size = 500
reset_on_extinction = True
[DefaultGenome]
node activataion options
activation_default = sigmoid
activation_mutate_rate = 0.0
activation_options = sigmoid
node aggregation options
aggregation_default = sum
aggregation_mutate_rate = 0.0
aggregation_options = sum
node bias options
bias_init_mean = 0.0
bias_init_stdev = 1.0
bias_max_value = 30.0
bias_min_value = -30.0
bias_mutate_power = 0.5
bias_mutate_rate = 0.7
bias_replace_rate = 0.1
genome compatibility options
compatibility_disjoint_coefficient = 1.1
compatibility_weight_coefficient = 0.5
connection add/reomve rates
conn_add_prob = 0.5
conn_delete_prob = 0.1
connection enable options
enabled_default = True
enabled_mutate_rate = 0.01
feed_forward = False
initial_connection = partial_direct 0.5
node add/reomve rates
node_add_prob = 0.1
node_delete_prob = 0.1
network parameters
num_hidden = 1
num_inputs = 10
num_outputs = 2
node responde options
response_init_mean = 1.0
response_init_stdev = 0.0
response_max_value = 30.0
response_min_value = -30.0
response_mutate_power = 0.0
response_mutate_rate = 0.0
response_replace_rate = 0.0
connection weight options
weight_init_mean = 0.0
weight_init_stdev = 1.0
weight_max_value = 30.0
weight_min_value = -30.0
weight_mutate_power = 0.5
weight_mutate_rate = 0.8
weight_replace_rate = 0.1
[DefaultSpeciesSet]
compatibility_threshold = 3.0
[DefaultStagnation]
species_fitness_func = max
max_stagnation = 100
species_elitism = 1
[DefaultReproduction]
elitism = 2
survival_threshold = 0.1
min_species_size = 2
- maze_environment.py
import math
import agent
import geometry
from novelty_archive import NoveltyItem
The maximal allowed speed for the maze solver agent
MAX_AGENT_SPEED = 3.0
def maze_novelty_metric(first_item, second_item):
“”"
The function to calculate the novelty metric score as a distance
between two data vectors in provided NoveltyItems
Arguments:
first_item: The first NoveltyItem
second_item: The second NoveltyItem
Returns:
The novelty metric as a distance between two data vectors
in provided noveltyItems
“”"
if not (hasattr(first_item, ‘data’) or hasattr(second_item, ‘data’)):
return NotImplemented
if len(first_item.data) != len(second_item.data):
can not be compared
return 0.0
diff_accum = 0.0
size = len(first_item.data)
for i in range(size):
diff = abs(first_item.data[i] - second_item.data[i])
diff_accum += diff
return diff_accum / float(size)
def maze_novelty_metric_euclidean(first_item, second_item):
“”"
The function to calculate the novelty metric score as a distance
between two data vectors in provided NoveltyItems.
Arguments:
first_item: The first NoveltyItem
second_item: The second NoveltyItem
Returns:
The novelty metric as a distance between two data vectors
in provided noveltyItems
“”"
if not (hasattr(first_item, ‘data’) or hasattr(second_item, ‘data’)):
return NotImplemented
if len(first_item.data) != len(second_item.data):
can’t be compared
return 0.0
diff_accum = 0.0
size = len(first_item.data)
for i in range(size):
diff = (first_item.data[i] - second_item.data[i])
diff_accum += (diff * diff)
return math.sqrt(diff_accum)
class MazeEnvironment:
“”"
This class encapsulates the maze simulation environment.
“”"
def init(self, agent, walls, exit_point, exit_range=5.0):
“”"
Creates new maze environment with specified walls and exit point.
Arguments:
agent: The maze novigating agent
walls: The maze walls
exit_point: The maze exit point
exit_range: The range arround exit point marking exit area
“”"
self.walls = walls
self.exit_point = exit_point
self.exit_range = exit_range
The maze navigating agent
self.agent = agent
The flag to indicate if exit was found
self.exit_found = False
The initial distance of agent from exit
self.initial_distance = self.agent_distance_to_exit()
The sample rate of agent position points saving during simulation steps.
self.location_sample_rate = -1
Update sensors
self.update_rangefinder_sensors()
self.update_radars()
def agent_distance_to_exit(self):
“”"
The function to estimate distance from maze solver agent to the maze exit.
Returns:
The distance from maze solver agent to the maze exit
“”"
return self.agent.location.distance(self.exit_point)
def test_wall_collision(self,loc):
“”"
The function to test if agent at specified location collides with any of
the maze walls
Argument:
loc: The new agent location to test for collision.
Returns:
The True if agent at new location will collide with any of the maze walls.
“”"
for w in self.walls:
if w.distance(loc) < self.agent.radius:
return True
return False
def create_net_inputs(self):
“”"
The function to create the ANN input values from the simulaiton environment.
Returns:
The list of ANN inputs consist of values get from solver agent sensors.
“”"
inputs = []
The range finders
for ri in self.agent.range_finders:
inputs.append(ri)
The radar sensors:
for rs in self.agent.radar:
inputs.append(rs)
return inputs
def apply_control_signals(self, control_signals):
“”"
The function to apply control signals received from control ANN to the
maze solver agent.
Arguments:
control_signals: The control received from the control ANN.
“”"
self.agent.angular_vel += (control_signals[0] - 0.5)
self.agent.speed += (control_signals[1] - 0.5)
constrain the speed & angular velocity
if self.agent.speed > MAX_AGENT_SPEED:
self.agent.speed = MAX_AGENT_SPEED
if self.agent.speed < -MAX_AGENT_SPEED:
self.agent.speed = -MAX_AGENT_SPEED
if self.agent.angular_vel > MAX_AGENT_SPEED:
self.agent.angular_vel = MAX_AGENT_SPEED
if self.agent.angular_vel < -MAX_AGENT_SPEED:
self.agent.angular_vel = -MAX_AGENT_SPEED
def update_rangefinder_sensors(self):
“”"
The function to update the agent range finder sensors.
“”"
for i, angle in enumerate(self.agent.range_finder_angles):
rad = geometry.deg_to_rad(angle)
project a point from agent location outwards
projection_point = geometry.Point(
x = self.agent.location.x + math.cos(rad) * self.agent.range_finder_range,
y = self.agent.location.y + math.sin(rad) * self.agent.range_finder_range
)
rotate the projection point by the agent’s heading angle to
algin it with heading direction.
projection_point.rotate(self.agent.heading, self.agent.location)
create the line segment from the agent location to the projected point
projection_line = geometry.Line(
a = self.agent.location,
b = projection_point
)
set range to maximum detection range
min_range = self.agent.range_finder_range
now test against maze walls to see if projection line hits any wall
and find the closest hit
for wall in self.walls:
found, intersection = wall.intersection(projection_line)
if found:
found_range = intersection.distance(self.agent.location)
we are interested in the closest hit
if found_range < min_range:
min_range = found_range
Update sendor value
self.agent.range_finders[i] = min_range
def update_radars(self):
“”"
The function to update the agent radar sensors.
“”"
target = geometry.Point(self.exit_point.x, self.exit_point.y)
rotate target with respect to the agent’s heading to align it
with haeading diretion
target.rotate(self.agent.heading, self.agent.location)
translate with respect to the agent’s location
target.x -= self.agent.location.x
target.y -= self.agent.location.y
the angle between maze eixt point and the agent’s heading direction
angle = target.angle()
find the appropriate radar sensor to be fired
for i, r_angles in enumerate(self.agent.radar_angles):
self.agent.radar[i] = 0.0 # reset specific radar
if (angle >= r_angles[0] and angle < r_angles[1]) or (angle + 360 >= r_angles[0] and angle + 360 < r_angles[1]):
self.agent.radar[i] = 1.0 # fire teh radar
def update(self,control_signals):
“”"
The function to update the solver agent position within maze.
After agent position updated it will be checked to find out if maze exit was
reached after that.
Arguments:
control_signals: The control signals received from control ANN
自我介绍一下，小编13年上海交大毕业，曾经在小公司待过，也去过华为、OPPO等大厂，18年进入阿里一直到现在。
深知大多数Python工程师，想要提升技能，往往是自己摸索成长或者是报班学习，但对于培训机构动则几千的学费，着实压力不小。自己不成体系的自学效果低效又漫长，而且极易碰到天花板技术停滞不前！
因此收集整理了一份《2024年Python开发全套学习资料》，初衷也很简单，就是希望能够帮助到想自学提升又不知道该从何学起的朋友，同时减轻大家的负担。
既有适合小白学习的零基础资料，也有适合3年以上经验的小伙伴深入学习提升的进阶课程，基本涵盖了95%以上Python开发知识点，真正体系化！
由于文件比较大，这里只是将部分目录大纲截图出来，每个节点里面都包含大厂面经、学习笔记、源码讲义、实战项目、讲解视频，并且后续会持续更新
如果你觉得这些内容对你有帮助，可以添加V获取：vip1024c （备注Python）
一、Python所有方向的学习路线
Python所有方向的技术点做的整理，形成各个领域的知识点汇总，它的用处就在于，你可以按照下面的知识点去找对应的学习资源，保证自己学得较为全面。
二、Python必备开发工具
工具都帮大家整理好了，安装就可直接上手！
三、最新Python学习笔记
当我学到一定基础，有自己的理解能力的时候，会去阅读一些前辈整理的书籍或者手写的笔记资料，这些笔记详细记载了他们对一些技术点的理解，这些理解是比较独到，可以学到不一样的思路。
四、Python视频合集
观看全面零基础学习视频，看视频学习是最快捷也是最有效果的方式，跟着视频中老师的思路，从基础到深入，还是很容易入门的。
五、实战案例
纸上得来终觉浅，要学会跟着视频一起敲，要动手实操，才能将自己的所学运用到实际当中去，这时候可以搞点实战案例来学习。
六、面试宝典
简历模板
一个人可以走的很快，但一群人才能走的更远。如果你从事以下工作或对以下感兴趣，欢迎戳这里加入程序员的圈子，让我们一起学习成长！
AI人工智能、Android移动开发、AIGC大模型、C C#、Go语言、Java、Linux运维、云计算、MySQL、PMP、网络安全、Python爬虫、UE5、UI设计、Unity3D、Web前端开发、产品经理、车载开发、大数据、鸿蒙、计算机网络、嵌入式物联网、软件测试、数据结构与算法、音视频开发、Flutter、IOS开发、PHP开发、.NET、安卓逆向、云计算
66129f47b8a67243c1008edf79.png)
既有适合小白学习的零基础资料，也有适合3年以上经验的小伙伴深入学习提升的进阶课程，基本涵盖了95%以上Python开发知识点，真正体系化！
由于文件比较大，这里只是将部分目录大纲截图出来，每个节点里面都包含大厂面经、学习笔记、源码讲义、实战项目、讲解视频，并且后续会持续更新
如果你觉得这些内容对你有帮助，可以添加V获取：vip1024c （备注Python）
[外链图片转存中…(img-Zf5uZlX7-1712301830366)]
一、Python所有方向的学习路线
Python所有方向的技术点做的整理，形成各个领域的知识点汇总，它的用处就在于，你可以按照下面的知识点去找对应的学习资源，保证自己学得较为全面。
二、Python必备开发工具
工具都帮大家整理好了，安装就可直接上手！
三、最新Python学习笔记
当我学到一定基础，有自己的理解能力的时候，会去阅读一些前辈整理的书籍或者手写的笔记资料，这些笔记详细记载了他们对一些技术点的理解，这些理解是比较独到，可以学到不一样的思路。
四、Python视频合集
观看全面零基础学习视频，看视频学习是最快捷也是最有效果的方式，跟着视频中老师的思路，从基础到深入，还是很容易入门的。
五、实战案例
纸上得来终觉浅，要学会跟着视频一起敲，要动手实操，才能将自己的所学运用到实际当中去，这时候可以搞点实战案例来学习。
六、面试宝典
简历模板
一个人可以走的很快，但一群人才能走的更远。如果你从事以下工作或对以下感兴趣，欢迎戳这里加入程序员的圈子，让我们一起学习成长！
AI人工智能、Android移动开发、AIGC大模型、C C#、Go语言、Java、Linux运维、云计算、MySQL、PMP、网络安全、Python爬虫、UE5、UI设计、Unity3D、Web前端开发、产品经理、车载开发、大数据、鸿蒙、计算机网络、嵌入式物联网、软件测试、数据结构与算法、音视频开发、Flutter、IOS开发、PHP开发、.NET、安卓逆向、云计算
642
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
