Value Function Approximation on Non-Linear Manifolds for Robot Motor Control
Robotics and Automation
View/ Open
Date
04/2007Author
Sugiyama, Masashi
Hachiya, Hirotaka
Towell, Christopher
Vijayakumar, Sethu
Metadata
Abstract
The least squares approach works efficiently in
value function approximation, given appropriate basis functions.
Because of its smoothness, the Gaussian kernel is a
popular and useful choice as a basis function. However, it
does not allow for discontinuity which typically arises in realworld
reinforcement learning tasks. In this paper, we propose
a new basis function based on geodesic Gaussian kernels,
which exploits the non-linear manifold structure induced by
the Markov decision processes. The usefulness of the proposed
method is successfully demonstrated in a simulated robot arm
control and Khepera robot navigation.