keras rl typeerror: len is not well defined for symbolic tensors. please call x.shape for shape info

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/python-3.7.6/lib/python3.7/site-packages/rl/agents/dqn.py", line 108, in __init__
  if hasattr(model.output, '__len__') and len(model.output) > 1:
File "/opt/python-3.7.6/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 733, in __len__
  "shape information.".format(self.name))
TypeError: len is not well defined for symbolic Tensors. (activation_4/Identity:0) Please call `x.shape` rather than `len(x)` for shape information.

I have fixed the error by changing the specified line of code (i.e., 108) with:

if hasattr(model.output, '__shape__') and len(model.output.shape) > 2:

I hope this is a correct fix.

All the best,
Ciprian

P.S. I use

Keras==2.3.1

with the backend

tensorflow==2.1.0
tensorflow-estimator==2.1.0

it works for me, i using tensorflow 1.15, and keras 2.