Cohen, Shay

Titov, Ivan

Balkır, Esma

2022-01-13T12:20:56Z

2022-01-13T12:20:56Z

2021-11-30

The task of knowledge graph (KG) completion, where one is given an incomplete KG as a list of facts, and is asked to give high scores to correct but unseen triples, has been a well-studied problem in the NLP community. A simple but surprisingly robust approach for solving this task emerged as learning low dimensional embeddings for entities and relations by approximating the underlying KG directly through a scoring function. Knowledge graphs have a natural representation as a binary three way array, also known as a 3rd order tensor, and certain classes of scoring functions can be characterized as finding a low-rank decomposition of this tensor. This dissertation extends this characterization, and investigates the suitability of tensors for modelling both knowledge graphs and related data, for learning low-rank representations of entities and relations that incorporate information from heterogeneous sources, and for reasoning with paths and rules using the learned representations. Specifically, we present two joint tensor decomposition models for integrating external information in the process of learning KG embeddings. Our first model is a joint tensor-tensor decomposition model that learns representations based on both KG facts and type information on entities and relations. Our second model is a joint tensor-matrix decomposition for integrating cooccurrence information between entities and words from an entity linked corpus into knowledge graph embeddings, in order to learn better representations for the entities that are rarely seen in the knowledge graph. We also investigate tensors as tools for enabling multi-step reasoning using learned embedding representations. To this end, we extend theoretical results for semiring weighted logic programs to tensors of semirings. Our results are broadly applicable to any area that uses dynamic programming algorithms for calculating tensor values. Such applications include incorporating embeddings of paths and rules for knowledge graph completion, and syntactic parsing with latent variable grammars

https://hdl.handle.net/1842/38412

http://dx.doi.org/10.7488/era/1677

en

The University of Edinburgh

E. Balkır, M. Naslidnyk, D. Palfrey and A. Mittal. Using pairwise occurrence information to improve knowledge graph completion on large-scale datasets. in the Proceedings of EMNLP-IJCNLP. 2019.

E. Balkır, M. Naslidnyk, D. Palfrey and A. Mittal. Improving knowledge graph embeddings with inferred entity types. Relational Representation Learning work-shop at NeurIPS. 2018.

E. Balkır, D. Gildea and S. Cohen. Tensors over Semirings for Latent-Variable Weighted Logic Programs. in the Proceedings of International Conference on Parsing Technologies (IWPT). 2020.

knowledge graphs

link prediction

embeddings

learned knowledge graph embeddings

knowledge graph completion

KG

natural language processing

NLP

joint tensor decomposition models

KG embeddings

Tensors and tensor decompositions for combining external information with knowledge graph embeddings

Thesis or Dissertation

Doctoral

PhD Doctor of Philosophy