github开源bark-ml:https://github.com/bark-simulator/bark-ml/
bark官网:https://bark-simulator.github.io/
bark-ml指导手册:https://bark-simulator.github.io/tutorials/bark_ml_getting_started/
Youtube视频:https://www.youtube.com/user/fortissTV/search?query=BARK
相关论文:
- Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments (IV 2020)
- BARK: Open Behavior Benchmarking in Multi-Agent Environments (IROS 2020)
- Counterfactual Policy Evaluation for Decision-Making in Autonomous Driving (IROS 2020, PLC Workshop)
一、项目下载
# 新建一个Anaconda环境
conda create -n bark-ml python=3.7
# 如果想删除环境采用以下操作
# conda remove -n your_env_name(虚拟环境名称) --all
# 进入环境
conda activate bark-ml
pip install virtualenv==16.7.8
pip install bark-ml
git clone https://github.com/bark-simulator/bark-ml
cd bark-ml
二、配置环境
1、修改install.sh
which python # 查看自己的python路径:/home/myx/anaconda3/envs/bark-ml/bin/python
sudo vim utils/install.sh # 修改install.sh文件
将python3.7
替换为刚刚得到的自己的python路径/home/myx/anaconda3/envs/bark-ml/bin/python
bash utils/install.sh
# 可能需要自己运行一下
# pip install torch==1.9.0+cpu torchvision -f https://download.pytorch.org/whl/torch_stable.html
# 进入虚拟环境
source utils/dev_into.sh
2、修改WORKSPACE
sudo gedit WORKSPACE
将原句
python_configure(name = “local_config_python”)
改为
python_configure(name = “local_config_python”, python_version = “3”)
三、测试
# source utils/dev_into.sh
# deactivate
bazel test //...
bazel test //bark_ml:unit_tests
bazel run //examples:continuous_env
bazel run //examples:blueprint_config
bazel run //examples:tfa
bazel run //examples:tfa_gnn -- --mode=train
bazel run //examples:tfa_counterfactual -- --mode=train
# 如果想训练
# bazel run //examples:tfa -- --mode=train
- 新建自己的参数文件夹 myx_params
/home/myx/Project/bark-ml/examples/myx_checkpoints/
/home/myx/Project/bark-ml/bark_ml/environments/blueprints/merging/merging.py
# 机器人行为和一大堆参数
/home/myx/anaconda3/envs/bark-ml/lib/python3.7/site-packages/tf_agents/drivers/dynamic_episode_driver.py
# 跑循环神经网络训练_loop_body_fn
/home/myx/Project/bark-ml/bark_ml/environments/blueprints/merging/merging.py
# evaluator 设置
/home/myx/Project/bark-ml/bark_ml/environments/single_agent_runtime.py
# reward, done, info = self._evaluator.Evaluate(
/home/myx/Project/bark-ml/bark_ml/evaluators/general_evaluator.py
def Evaluate(self, observed_world, action):
"""Returns information about the current world state."""
eval_results = observed_world.Evaluate()
reward = 0.
scheduleTerminate = False
for _, eval_fn in self._bark_ml_eval_fns.items():
t, r, i = eval_fn(observed_world, action, eval_results)
eval_results = {**eval_results, **i} # merge info
reward += r # accumulate reward
if t: # if any of the t are True -> terminal
scheduleTerminate = True
return reward, scheduleTerminate, eval_results
/home/myx/anaconda3/envs/bark-ml/lib/python3.7/site-packages/tf_agents/agents/ppo/ppo_agent.py
# ppo 训练代码