- Rolf, M., "Goal Babbling with Unknown Ranges: A Direction-Sampling Approach", and
- Reinhart, R.F., and M. Rolf, "Learning Versatile Sensorimotor Coordination with Goal Babbling and Neural Associative Dynamics"
|Discovery with Goal Babbling|
|In contrast: random motor babbling|
When exploring goals, one needs to know what goals explore. That's often not quite trivial, in particular if one does not even know what goals can be achieved at all. Identifying the range of what can be achieved by means of motor commands (e.g. set of hand positions by means of postures) is a generally important thing. For goal babbling it is even more important since goal babbling starts by selecting goals, ideally from this set of achievable outcomes. The first paper introduces a novel approach to tackle this problem. The proposed solution is based on, well, a direction sampling of goals (as one might guess from the title of the paper).
Abstract — Goal babbling is a recent concept for the efficient bootstrapping of sensorimotor coordination that is inspired by infants’ early goal-directed movement attempts. Several studies have shown its superior performance compared to random motor babbling. Yet, previous implementations of goal babbling require knowledge of a set of achievable goals in advance. This paper introduces an approach to goal babbling that can bootstrap coordination skills without pre-specifying, or even representing, a set of goals. On the contrary, it can discover the ranges of achievable goals autonomously. This capability is demonstrated in a challenging task with up to 50 degrees of freedom, in which the discovery of possible outcomes is shown to be desperately intractable with random motor babbling.
In the second paper, "Learning Versatile Sensorimotor Coordination with Goal Babbling and Neural Associative Dynamics", Felix Reinhart and myself propose a new way to organize sensorimotor learning and coordination by combining several approaches in a neat and very flexible manner. Dissecting the title, we first of all use goal babbling, which is known to allow for very efficient learning. Goal babbling typically explores a single way to solve some sensorimotor task (that's why it can be so efficient). Often, one solution is enough. But not always. The neat thing is: we show that goal babbling allows for a decent control about which solution is learned. Thus, one can also explore a discrete number of different solutions, which is still much, much more efficient than exploring everything. The key question is how to combine and select such solutions (see image at the top). That's the clue. As the title suggests, our solution is something about neural networks...
Abstract — We combine an efficient exploration mechanism for bootstrapping of sensorimotor coordination skills with an associative network model to represent multiple coordination styles. Both approaches are integrated into a three-phased process of exploration, i.e. discovery of a new model, consolidation, the long-term storage of multiple models in a dynamical associative network, and exploitation of multiple models by the neural dynamics for versatile sensorimotor coordination. The proposed exploration-consolidation process is demonstrated for a planar robotic manipulator with ten degrees of freedom. Exploitation of sensorimotor coordination from the consolidated neural dynamics features motor hysteresis and additionally comprises a forward model that can be utilized to interpret proprioceptive feedback.
You can get the entire story by listening to our talks at the conference. Next stop: Osaka!