Leather, Hugh

Petoumenos, Pavlos

Mayr, Richard

Cummins, Christopher Edward

Engineering and Physical Sciences Research Council (EPSRC)

2020-03-12T12:08:35Z

2020-03-12T12:08:35Z

2020-06-25

Constructing compilers is hard. Optimising compilers are multi-million dollar projects spanning years of development, yet remain unable to fully exploit the available performance, and are prone to bugs. The rapid transition to heterogeneous parallelism and diverse architectures has raised demand for aggressively-optimising compilers to an all time high, leaving compiler developers struggling to keep up. What is needed are better tools to simplify compiler construction. This thesis presents new techniques that dramatically lower the cost of compiler construction, while improving robustness and performance. The enabling insight for this research is the leveraging of deep learning to model the correlations between source code and program behaviour, enabling tasks which previously required significant engineering effort to be automated. This is demonstrated in three domains: First, a generative model for compiler benchmarks is developed. The model requires no prior knowledge of programming languages, yet produces output of such quality that professional software developers cannot distinguish generated from hand-written programs. The efficacy of the generator is demonstrated by supplementing the training data of predictive models for compiler optimisations. The generator yields an automatic improvement in heuristic performance, and exposes weaknesses in state-of-the- art approaches which, when corrected, yield further performance improvements. Second, a compiler fuzzer is developed which is far simpler than prior techniques. By learning a generative model rather than engineering a generator from scratch, it is implemented in 100 fewer lines of code than the state-of-the-art, yet is capable of exposing bugs which prior techniques cannot. An extensive testing campaign reveals 67 new bugs in OpenCL compilers, many of which have now been fixed. Finally, this thesis addresses the challenge of feature design. A methodology for learning compiler heuristics is presented that, in contrast to prior approaches, learns directly over the raw textual representation of programs. The approach outperforms state-of-the-art models with hand-engineered features in two challenging optimisation domains, without requiring any expert guidance. Additionally, the methodology enables models trained in one task to be adapted to perform another, permitting the novel transfer of information between optimisation problem domains. The techniques developed in these three contrasting domains demonstrate the exciting potential of deep learning to simplify and improve compiler construction. The outcomes of this thesis enable new lines of research to equip compiler developers to keep up with the rapidly evolving landscape of heterogeneous architectures.

https://hdl.handle.net/1842/36866

http://dx.doi.org/10.7488/era/168

en

The University of Edinburgh

Chris Cummins, Pavlos Petoumenos, Zheng Wang, and Hugh Leather “Synthesizing Benchmarks for Predictive Modeling”. In Proceedings of the International Symposium on Code Generation and Optimization (CGO), 2017.

Chris Cummins, Pavlos Petoumenos, Alastair Murray, and Hugh Leather “Compiler Fuzzing through Deep Learning”. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2018.

Chris Cummins, Pavlos Petoumenos, Zheng Wang, and Hugh Leather “End-to-end Deep Learning of Optimization Heuristics”. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT), 2017.

C. Cummins, P. Petoumenos, M. Steuwer, and H. Leather. “Autotuning OpenCL Workgroup Size for Stencil Patterns”. In: International Workshop on Adaptive Self-tuning Computing Systems (ADAPT). 2016.

C. Cummins, P. Petoumenos, M. Steuwer, and H. Leather. “Towards Collaborative Performance Tuning of Algorithmic Skeletons”. In: High- Level Programming for Heterogeneous and Hierarchical Parallel Systems (HLPGPU). 2016.

C. Cummins, P. Petoumenos, A. Murray, and H. Leather. “DeepSmith: Compiler Fuzzing through Deep Learning”. In: ACACES. 2018.

optimising compilers

compiler construction

deep learning

generative model

compiler fuzzer

compiler heuristics

Deep learning for compilers

Thesis or Dissertation

Doctoral

PhD Doctor of Philosophy