Multi-task dynamical systems: customising time series models
Abstract
Time series datasets are usually composed of a variety of sequences from the
same domain, but from different entities, such as individuals, products, or
organizations. We are interested in how time series models can be specialized to
individual sequences (capturing the specific characteristics) while still retaining
statistical power by sharing commonalities across the sequences.
The major contribution of this thesis is the multi-task dynamical system
(MTDS); a general methodology for extending multi-task learning (MTL) to
time series models. Our approach endows dynamical systems with a set of hierarchical latent variables which can modulate all model parameters. To our
knowledge, this is a novel development of MTL, and applies to time series both
with and without control inputs. This thesis provides a formulation of three
example MTDS models: a multi-task linear dynamical system, a multi-task
recurrent neural network (RNN) and a multi-task pharmacodynamic model.
The use of MTDS models is demonstrated in detail on three datasets. For a
synthetic physical dataset, we demonstrate that our learning and inference algorithms are able to recover a model that is indistinguishable from the optimal
one, and provide substantial improvements in prediction over gated RNN models. We apply the MTDS to motion-capture data of people walking in various
styles, which results in improved performance, style transfer and the ability
to morph between walking styles. Finally, using patient drug-response data,
we show that our MTDS approach can result in significant improvements over
modern pharmacodynamic (PD) models without deviating from the trusted
PD model form.