Multi-task dynamical systems: customising time series models

Abstract

Time series datasets are usually composed of a variety of sequences from the same domain, but from different entities, such as individuals, products, or organizations. We are interested in how time series models can be specialized to individual sequences (capturing the specific characteristics) while still retaining statistical power by sharing commonalities across the sequences. The major contribution of this thesis is the multi-task dynamical system (MTDS); a general methodology for extending multi-task learning (MTL) to time series models. Our approach endows dynamical systems with a set of hierarchical latent variables which can modulate all model parameters. To our knowledge, this is a novel development of MTL, and applies to time series both with and without control inputs. This thesis provides a formulation of three example MTDS models: a multi-task linear dynamical system, a multi-task recurrent neural network (RNN) and a multi-task pharmacodynamic model. The use of MTDS models is demonstrated in detail on three datasets. For a synthetic physical dataset, we demonstrate that our learning and inference algorithms are able to recover a model that is indistinguishable from the optimal one, and provide substantial improvements in prediction over gated RNN models. We apply the MTDS to motion-capture data of people walking in various styles, which results in improved performance, style transfer and the ability to morph between walking styles. Finally, using patient drug-response data, we show that our MTDS approach can result in significant improvements over modern pharmacodynamic (PD) models without deviating from the trusted PD model form.