Inferential automation of statistical methodologies in a query-answering system

Abstract

Query Answering (QA) systems are the most broadly accessible interface to data-focussed systems, most effectively hiding high inner complexity, with their main use being in data retrieval, usually with minimal inference. The automation of more sophisticated data processing, the realm of Intelligent Data/Discovery Assistants and AutoML, is a natural candidate for combination with such a useful interface.

FRANK is a multi-domain QA system which performs inferential and simple statistical reasoning on data which it automatically identifies and retrieves (e.g. to provide predictions). SMART is a statistics advisor system designed to allow FRANK to answer queries of types which have not previously been within the purview of QA systems, including statistical significance, functional shape description, estimation of model response probability, and analysis of variance.

The combined SMART FRANK system allows users to quickly obtain answers to data-focussed questions involving choices of analysis, modelling and visualisation which are tedious and time-consuming for an expert and beyond the knowledge of a novice. This expands the reach of automated inference of information not available in knowledge-bases and can be a useful tool in identifying features and questions of interest in the preliminary overview of a given data domain.

The aim of this research is not to replace proper data investigation, but instead to lower the barriers of effort and skill so as to not prematurely cut off inspiration and momentum in investigating data-focussed questions, giving new capabilities to some and saving the time of others.