Signature methods in machine learning

Signature-based techniques give mathematical insight into the interactions between complex streams of evolving data. These insights can be quite naturally translated into numerical approaches to understanding streamed data, and perhaps because of their mathematical precision, have proved useful in analysing streamed data in situations where the data is irregular, and not stationary, and the dimension of the data and the sample sizes are both moderate. Understanding streamed multi-modal data is exponential: a word in n letters from an alphabet of size d can be any one of d^{n} messages. Signatures provide a “lossy compression” of the information contained within such a stream by filtering out the parameterisation noise. More concretely, suppose we have a time series with 3 channels and N samples. There are 1 + 3N + \frac{3N(3N+1)}{2}= 1 + \frac{9}{2}N(N+1) linearly independent quadratic polynomials defined on the time series. But the signature of this time series truncated to depth 2 only consists of 1 + 3 + 3^{2} = 13 components which is, in particular, independent of the number of samples N . However, whilst the independence of the number of samples N has removed an exponential amount of noise, the dependence on the number of channels to the power of the depth ensures that an exponential amount of information remains. This survey aims to stay in the domain where that exponential scaling can be managed directly. Scalability issues are an important challenge in many problems but would require another survey article and further ideas. This survey describes a range of contexts where the data sets are small and the existence of small sets of context free and principled features can be used effectively. The mathematical nature of the tools can make their use intimidating to non-mathematicians. The examples presented in this article are intended to bridge this communication gap and provide tractable working examples drawn from the machine learning context. Notebooks are available online for several of these examples. This survey builds on the earlier paper of Ilya Chevryev and Andrey Kormilitzin which had broadly similar aims at an earlier point in the development of this machinery. This article illustrates how the theoretical insights offered by signatures are simply realised in the analysis of application data in a way that is largely agnostic to the data type. Larger and more complex problems would expect to address scalability issues and draw on a wider range of data science techniques. The article starts with a brief discussion of background material related to machine learning and signatures. This discussion fixes notation and terminology whilst simplifying the dependencies, but these background sections are not a substitute for the extensive literature they draw from. Hopefully, by working some of the examples, the reader will find access to useful and simple to deploy tools; tools that are moderately effective in analysing longitudinal data that is complex and irregular in contexts where massive machine learning is not a possibility.