使用DQN问题整理

已于 2022-11-18 18:38:11 修改
阅读量724 收藏 4
点赞数 1
分类专栏： 深度学习 文章标签： python tensorflow 深度学习
于 2022-11-18 17:16:19 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_39698985/article/details/127922370
版权

一、原始github地址

https://github.com/devsisters/DQN-tensorflow

二、安装环境

- python  3.7.13
- gym 0.15.7
- tqdm 4.61.2
- tensorflow 1.15.0

三. 相关问题

问题1

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 3, in <module>
    import tensorflow as tf
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow/__init__.py", line 99, in <module>
    from tensorflow_core import *
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/__init__.py", line 28, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "<frozen importlib._bootstrap>", line 1019, in _handle_fromlist
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow/__init__.py", line 50, in __getattr__
    module = self._load()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow/__init__.py", line 44, in _load
    module = _importlib.import_module(self.__name__)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/core/framework/graph_pb2.py", line 16, in <module>
    from tensorflow.core.framework import node_def_pb2 as tensorflow_dot_core_dot_framework_dot_node__def__pb2
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/core/framework/node_def_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/core/framework/tensor_shape_pb2.py", line 42, in <module>
    serialized_options=None, file=DESCRIPTOR),
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Process finished with exit code 1

解决方案

pip install 'protobuf~=3.19.0'

问题2

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/dqn/utils.py", line 11, in <module>
    from scipy.misc import imresize
ModuleNotFoundError: No module named 'scipy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 5, in <module>
    from dqn.agent import Agent
  File "/home/user/code/python/DQN-tf1.15/dqn/agent.py", line 11, in <module>
    from .replay_memory import ReplayMemory
  File "/home/user/code/python/DQN-tf1.15/dqn/replay_memory.py", line 8, in <module>
    from .utils import save_npy, load_npy
  File "/home/user/code/python/DQN-tf1.15/dqn/utils.py", line 13, in <module>
    import cv2
ModuleNotFoundError: No module named 'cv2'

Process finished with exit code 1

解决方案

pip install opencv-python

问题3

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 49, in main
    config = get_config(FLAGS) or FLAGS
  File "/home/user/code/python/DQN-tf1.15/config.py", line 58, in get_config
    for k, v in FLAGS.__dict__['__flags'].items():
KeyError: '__flags'

Process finished with exit code 1

解决方案
将

for k, v in FLAGS.__dict__['__flags'].items():

替换为

for k in FLAGS:
    v = FLAGS[k].value

问题4

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 54, in main
    env = GymEnvironment(config)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 67, in __init__
    super(GymEnvironment, self).__init__(config)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 8, in __init__
    self.env = gym.make(config.env_name)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 569, in make
    _check_version_exists(ns, name, version)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 219, in _check_version_exists
    _check_name_exists(ns, name)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 198, in _check_name_exists
    f"Environment {name} doesn't exist{namespace_msg}. {suggestion_msg}"
gym.error.NameNotFound: Environment Breakout doesn't exist. 

Process finished with exit code 1

解决方案

pip install ale-py
pip install gym[accept-rom-license]

问题5

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 57, in main
    raise Exception("use_gpu flag is true when no GPUs are available")
Exception: use_gpu flag is true when no GPUs are available

Process finished with exit code 1

解决方案

flags.DEFINE_boolean('use_gpu', True, 'Whether to use gpu or not')
## 修改为
flags.DEFINE_boolean('use_gpu', False, 'Whether to use gpu or not')

问题6

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 62, in main
    agent = Agent(config, env, sess)
  File "/home/user/code/python/DQN-tf1.15/dqn/agent.py", line 30, in __init__
    self.build_dqn()
  File "/home/user/code/python/DQN-tf1.15/dqn/agent.py", line 201, in build_dqn
    self.l3_flat = tf.reshape(self.l3, [-1, reduce(lambda x, y: x * y, shape[1:])])
NameError: name 'reduce' is not defined

Process finished with exit code 1

解决方案

from functools import reduce

问题7

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 62, in main
    agent = Agent(config, env, sess)
  File "/home/user/code/python/DQN-tf1.15/dqn/agent.py", line 31, in __init__
    self.build_dqn()
  File "/home/user/code/python/DQN-tf1.15/dqn/agent.py", line 228, in build_dqn
    for idx in xrange(self.env.action_size):
NameError: name 'xrange' is not defined

Process finished with exit code 1

解决方案

xrange 
# 替换为
range

问题8

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 62, in main
    agent = Agent(config, env, sess)
  File "/home/user/code/python/DQN-tf1.15/dqn/agent.py", line 31, in __init__
    self.build_dqn()
  File "/home/user/code/python/DQN-tf1.15/dqn/agent.py", line 329, in build_dqn
    self._saver = tf.train.Saver(self.w.values() + [self.step_op], max_to_keep=30)
TypeError: unsupported operand type(s) for +: 'dict_values' and 'list'

Process finished with exit code 1

解决方案

self._saver = tf.train.Saver(self.w.values() + [self.step_op], max_to_keep=30)
# 替换为
self._saver = tf.train.Saver(list(self.w.values()) + [self.step_op], max_to_keep=30)

问题9

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 65, in main
    agent.train()
  File "/home/user/code/python/DQN-tf1.15/dqn/agent.py", line 42, in train
    screen, reward, action, terminal = self.env.new_random_game()
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 28, in new_random_game
    self.new_game(True)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 23, in new_game
    self._step(0)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 35, in _step
    self._screen, self.reward, self.terminal, _ = self.env.step(action)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/wrappers/order_enforcing.py", line 36, in step
    raise ResetNeeded("Cannot call env.step() before calling env.reset()")
gym.error.ResetNeeded: Cannot call env.step() before calling env.reset()

Process finished with exit code 1

解决方案

# 在environment.py文件中的def init语句内把
self._screen = None
# 更改为
self._screen = self.env.reset()

问题10

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 65, in main
    agent.train()
  File "/home/user/code/python/DQN-tf1.15/dqn/agent.py", line 42, in train
    screen, reward, action, terminal = self.env.new_random_game()
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 28, in new_random_game
    self.new_game(True)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 23, in new_game
    self._step(0)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 35, in _step
    self._screen, self.reward, self.terminal, _ = self.env.step(action)
ValueError: too many values to unpack (expected 4)

Process finished with exit code 1

解决方案

pip install gym==0.15.7

问题11

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 54, in main
    env = GymEnvironment(config)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 67, in __init__
    super(GymEnvironment, self).__init__(config)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 8, in __init__
    self.env = gym.make(config.env_name)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 156, in make
    return registry.make(id, **kwargs)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 101, in make
    env = spec.make(**kwargs)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 72, in make
    cls = load(self.entry_point)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 17, in load
    mod = importlib.import_module(mod_name)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/atari/__init__.py", line 1, in <module>
    from gym.envs.atari.atari_env import AtariEnv
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/atari/atari_env.py", line 13, in <module>
    "'pip install gym[atari]'.)".format(e))
gym.error.DependencyNotInstalled: No module named 'atari_py'. (HINT: you can install Atari dependencies by running 'pip install gym[atari]'.)

Process finished with exit code 1

解决方案

pip install gym[atari]

问题12

Traceback (most recent call last):
  File "/home/user/code/python/DQN-tf1.15/main.py", line 70, in <module>
    tf.app.run()
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/home/user/code/python/DQN-tf1.15/main.py", line 54, in main
    env = GymEnvironment(config)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 67, in __init__
    super(GymEnvironment, self).__init__(config)
  File "/home/user/code/python/DQN-tf1.15/dqn/environment.py", line 8, in __init__
    self.env = gym.make(config.env_name)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 156, in make
    return registry.make(id, **kwargs)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 101, in make
    env = spec.make(**kwargs)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/registration.py", line 73, in make
    env = cls(**_kwargs)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/gym/envs/atari/atari_env.py", line 49, in __init__
    self.game_path = atari_py.get_game_path(game)
  File "/home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/atari_py/games.py", line 20, in get_game_path
    raise Exception('ROM is missing for %s, see https://github.com/openai/atari-py#roms for instructions' % (game_name,))
Exception: ROM is missing for breakout, see https://github.com/openai/atari-py#roms for instructions

Process finished with exit code 1

解决方案

# 下载该文件
wget http://www.atarimania.com/roms/Roms.rar
# 解压该文件,解压后目录ROMS的路径为/home/user/code/python/DQN-tensorflow/Roms/ROMS
├── Roms
│   ├── HC ROMS
│   └── ROMS
# 进入安装包目录
cd /home/user/.conda/envs/tf_1.15/lib/python3.7/site-packages/
# 运行以下命令
python -m atari_py.import_roms /home/user/code/python/DQN-tensorflow/Roms/ROMS

四、修改后代码

https://github.com/Rory602/DQN-tf1.15

静心问道
关注
专栏目录
DQN-TC搭建整理
m0_46482145的博客
04-22 331
算法伪代码： M：训练模型时遍历训练集的次数 N：训练集中类别的数量 s[t] = [ x[1] ,a[1],x[2],a[2],x[3],a[3],...,a[t-1], x[t] ] 当前的文本是当前episode的最后一个的时候r[j]= EG ，否则r[j] = 0 【EG：推文的期望增益】 N : 文本数量，G(t) ： 文本的增益 具体流程： 1、使用单层LSTM层作为编码器，用于生成文本的向量表示 2、强化学习中使用三层架构的神经网络来近似估计Q值 ...
莫烦强化学习笔记整理（五） DQN-part2
lxs3213196的博客
11-11 1399
莫烦强化学习笔记整理（四） DQN-part2openAI gym环境库Double DQNDQN with Prioritized ReplayDueling DQN openAI gym环境库 Double DQN DQN with Prioritized Replay Dueling DQN
DQN代码问题
AI_future的博客
07-27 429
1.env.render()函数用于渲染出当前的智能体以及环境的状态。 2.env.reset()为重新初始化函数 3.observation_, reward, done = env.step(action) 第一个为当前屏幕图像的像素值，经过彩色转灰度、缩放等变换最终送入我们上一篇文章中介绍的 CNN 中，得到下一步“行为”； 第二个值为奖励，每当游戏得分增加时，该返回值会有一个正反馈； 第三个值 gameover 为布尔值，如果游戏结束，返回 True； ...
解决运行DQN小demo，tensorflow各种报错问题！
com_fang_bean的博客
04-23 258
需要使用tencorflow1.14.0,其他版本的2.0.0或者1.4.0都不行。
ubuntu16.4安装gym以及出现的问题
weixin_48702455的博客
05-17 1415
输入创建gym环境（gym是我自己设置的环境名字）出现这样界面之后选y 就开始创建了 现在只需等待 创建成果界面如下创建完输入激活gym环境 前面标签会由base变为gym（自己设置的名字）下一步输入下载安装提示安装成功后 通过（打开文件）（将这句添到文件末尾，保存文件就得了，home/gym是gym的存储路径）配置环境变量。
gym不存在环境解决办法
qq_36178962的博客
03-06 9678
最近开始学习强化学习，尝试使用gym训练一些小游戏，发现一直报环境不存在的问题，看到错误提示全是什么不存在环境，去官网以及github找了好几圈，贴过来的代码都用不了，后来发现是版本变迁，环境被移除了，我。。。 这里找到一个解决办法，重新安装旧版本的，能用就行，凑合着用 这是原博客的说明 版本太新了，gym库应该在0.20.0版本不支持atari，并在0.21.0版中连环境也没了[1] 直接在命令行安装 pip install gym==0.19.0 pip install atari_py==0.2.
DQN（deep Q-network）算法简述
星海浮生
03-11 5万+
基本概念；进阶技巧；连续动作的场景
莫烦强化学习笔记整理（四） DQN-part1
lxs3213196的博客
11-05 1335
莫烦强化学习笔记整理（四） DQN-part1什么是DQNDQN算法更新DQN神经网络DQN思维决策openAI gym环境库Double DQNDQN with Prioritized ReplayDueling DQN DQN-part1) 什么是DQN DQN算法更新 DQN神经网络 DQN思维决策 openAI gym环境库 Double DQN DQN with Prioritized Replay Dueling DQN ...
用keras搭建DQN
senjie_wang的博客
09-14 9046
dqn总结： 最近时间虽然一直看这个dqn代码，虽然看懂了，但是思路却一直不明确，无法自己复写出自己的代码，所以就想写一篇博客来整理一下思路。如果这篇文章写完了，应该思路就清晰了。 本文主要是看了莫烦的教学视频，在视频中，老师（尊称一声老师O(∩_∩)O哈哈~）是用tensoflow1.2来搭建框架的。本人在看懂了代码后，认为DQN中采用了两个模型，此时用tensorflow来完成这些网络的灵...
1-baselines/run.py解读
qq_45880533的博客
11-16 1580
1-baselines/run.py解读前言run.py参数设置main函数 前言 就个人来说，比较喜欢从main函数入手，根据其运行的流程，一步一步找到每一步中涉及的参数以及函数的用意，最终摸清整个一个项目的流程和框架。 书接上文，上文书说到我们根据GitHub上的提示，可以使用下面的命令成功的运行了一个例子。 python -m baselines.run --alg=deepq --env=PongNoFrameskip-v4 --num_timesteps=1e6 于是我们顺藤摸瓜，找到了bas
TensorFlow遇到的各种错误
天边一坨浮云
06-01 2493
TensorFlow遇到的各种错误
解决python导包报错ImportError: cannot import name ‘xxx‘ from
热门推荐
空山新雨后，天气晚来秋
02-18 5万+
花了较长时间，最后发现循环引用导致。既然循环引用了，只要让让它们别循环引用就行。 那么解决掉循环引用的部分即可正常！ 解决办法：1，拆解成多个文件，将要互相引用的分开 2，缩小导入的范围，如一般情况下import在文件顶部，这时候可以将import移动到紧挨需要导入的代码上方即可； ...
运行出错：ImportError: cannot import name ‘get_config‘ from ‘tensorflow.python.eager.context‘ (C:\Program
weixin_55511645的博客
03-08 3436
运行：from keras.models import Sequential 出错：ImportError: cannot import name 'get_config' from 'tensorflow.python.eager.context' 改为：from tensorflow.keras.models import Sequential就好了
实验室linux服务器安装gym出现的问题
xiaochen_hzau的博客
10-12 2618
实验室linux服务器安装gym出现的问题。还没解决，待续
DQN算法详解
Sweeping_Robot的博客
04-17 4780
强化学习算法可以分为三大类：value based, policy based 和 actor critic。常见的是以DQN为代表的value based算法，这种算法中只有一个值函数网络，没有policy网络，以及以DDPG,TRPO为代表的actor-critic算法，这种算法中既有值函数网络，又有policy网络。说到DQN中有值函数网络，这里简单介绍一下强化学习中的一个概念，叫值函数近似。一个state action pair 对应一个值函数。
gym环境错误：gym.error.UnregisteredEnv: No registered env with id: BipedalWalker-v2
Marshal Zheng的博客
03-15 1万+
gym环境错误：gym.error.UnregisteredEnv: No registered env with id: BipedalWalker-v2 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "TD3_Biped...
微信小程序云开发环境变动时要注意的问题Environment not found错误
李小砍的技术博客
12-18 4471
如果微信小程序启用云开发环境，会默认创建一个环境，在执行cloud.init操作时无需再添加环境名，但如果要更换环境，就需要更改代码了，否则会一直报错 Environment not found, there is no default environment exists, 原来的代码 const cloud = require('wx-server-sdk') cloud.init() const db = cloud.database() wx.cloud.callFunction({
基于深度Q学习的雅达利打砖块游戏博弈
学习日常分享
06-25 1729
球碰到砖块、棒子与底下以外的三边会反弹，落到底下会失去一颗球，把砖块全部消去就可以破关。 始祖是史蒂夫·乔布斯与他的好友沃兹（苹果公司的另一位创始人）于1975年的夏末，花了4天时间设计完成的游戏《乒乓》。同时，美国英宝格公司（en:Atari Games，ja:アタリ (ゲーム)）于1976年推出的街机游戏“Breakout”，由该公司在1972年发行的“PONG”（en:PONG，ja:ポン (ゲーム)，世界上第一款电子游戏，类似台球）改良而来。相较于其前作，一个人就可以玩与变化丰富这两项特点让Brea
使用DQN求解TSP
最新发布
12-05
使用DQN求解TSP的过程可以分为以下几个步骤： 1.定义状态空间和动作空间：在TSP问题中，状态空间可以定义为当前已经访问的城市集合，动作空间可以定义为下一个要访问的城市。 2.定义奖励函数：在TSP问题中，奖励函数可以定义为当前已经访问的城市数量的倒数，即访问的城市数量越多，奖励越小。 3.构建神经网络：使用DQN算法需要构建一个神经网络，用于估计每个状态下每个动作的Q值。 4.初始化神经网络参数：初始化神经网络的参数，可以使用随机初始化或者预训练的方式。 5.定义经验回放缓存：使用经验回放缓存来存储智能体的经验，用于训练神经网络。 6.定义探索策略：在训练过程中，需要使用一定的探索策略来保证智能体能够探索到更多的状态和动作。 7.训练神经网络：使用DQN算法来训练神经网络，不断更新神经网络的参数，使得神经网络能够更好地估计每个状态下每个动作的Q值。 8.测试神经网络：使用训练好的神经网络来解决TSP问题，不断输入当前状态，根据神经网络的输出选择下一个要访问的城市，直到所有城市都被访问过。
评论 3
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值