Automous Identification and Goal-directed Invocation of Event-Predictive Behavioral Primitives

Motivation

The problem of learning meaningful, compositional abstractions from sensorimotor experiences remains an open challenge. A large challenge for the brain as well as artificial cognitive systems lies in the effective segmentation of our continuous perceptual stream of sensorimotor information into such behavioral primitives.
In all cases, the beginning and end of a movement primitive is predefined and not autonomously discovered by the system itself. Furthermore, initially, the systems do not learn via self-exploration but typically from demonstrations.

Terminology

• sensorimotor
  sensorimotor是刺激作用于感觉神经而传至大脑，再由运动神经作出动作的活动。
  of, relating to, or functioning in both sensory and motor aspects of bodily activity
• building blocks of behavior or behavioral primitives
  Humans seem to organize our behavior and the accompanying perception into small, compositional structures in a highly systematic manner. These structures can be viewed as elementary units of behavior above the level of single motor commands.

behavioral primitive是本文解决问题的核心

Related work

In most cognitive systems approaches so far, behavioral primitives are segmented by hand, pre-programmed into the system, or learned by demonstration. In all these cases, though, the primitives are made explicit to the system, that is, the learning system does not need to identify the primitives autonomously.

Contribution

Introduce a computational learning architecture, termed surprise-based behavioral modularization
into event-predictive structures (SUBMODES), that learns behavioral primitives as well as behavioral transitions completely from scratch. 机器人通过self-explore得到movement而不是通过predefine或from demonstrations

Starting with this self-exploration mechanism, the algorithm learns internal models that are trained to predict the motor commands and the resulting sensory consequences of the currently performed behavior.

It has been suggested that our ability to serially combine these compositional elements is crucial for our ability to quickly learn complex motor skills and to flexibly adjust our behavior to new tasks.

Discovering behavioral primitives and applying them for high-level goal-directed control is closely related to hierarchical reinforcement learning and options framework.