Doumas, Alex

Rabagliati, Hugh

Shurkova, Ekaterina Y.

2022-11-17T16:52:30Z

2022-11-17T16:52:30Z

2022-11-17

Humans regularly reason from visual information, engaging in simple object search in a scene to abstract mathematical thinking. In recent decades, the field of machine learning has extensively focused on visual tasks with the aim to model human visual reasoning. However, machine learning approaches still do not match human performance on simple visual tasks such as the Synthetic Visual Reasoning Test (SVRT; Fleuret et al. 2011). While this set of tasks is trivial for humans to solve, the current state-of-the-art machine learning algorithms struggle with the SVRT. We argue that the reason for the difference in human reasoning and machines’ performance in the SVRT is the ways humans and machines represent the world and visual information specifically. We argue that humans represent situations in terms of relations between constituent objects, and that our representation of these relations is structured and symbolic. By consequence, humans engage in operations that are not available for machine systems that rely on non-structured representations. We hold that operations over structured relational representations is what underlie phenomena such as abstract visual reasoning and cross-domain generalisation. The current work builds on the DORA (Discovery Of Relations by Analogy; Doumas et al., 2008; 2022) model of relation learning. DORA learns structured representations of magnitude relations from simple visual inputs. Here we expand the model to learn more complex categorical relations (e.g., contains or supports) as compressions of simpler relations (e.g., above, in-contact), and develop a new method for identifying relevant relations over which to perform reasoning from simple scenes. We embed the resulting model in a pipeline for human visual reasoning consisting of successful psychological models of object recognition and analogy making. The result is an end-to-end system which is constrained as much as possible by what is known about the processes and mechanisms of the cognitive system–from early vision to learning complex relations and reasoning. The model is tested within the context of the SVRT. The limitations of the model and the directions for future research are discussed.

https://hdl.handle.net/1842/39495

http://dx.doi.org/10.7488/era/2745

en

The University of Edinburgh

Shurkova, E. Y. & Doumas, L. A. A. (2020). Simulating feature- and relation based categorisation with a symbolic-connectionist model. In S. Denison., M. Mack, Y. Xu, & B.C. Armstrong (Eds.), Proceedings of the 42nd Annual Conference of the Cognitive Science Society (pp. 3412- 3418). Cognitive Science Society

Shurkova, E. Y., & Doumas, L. A. A. (2021). Compression: A lossless mechanism for learning complex structured relational representations. Proceedings of the Annual Meeting of the Cognitive Science Society, 43. Retrieved from https://escholarship.org/uc/item/27287461

Shurkova, E. Y., & Doumas, L. A. A. (2022). Towards a model of visual reasoning. Proceedings of the Annual Meeting of the Cognitive Science Society, 44. Retrieved from https://escholarship.org/uc/item/3tc6256x

deep neural networks

Synthetic Visual Reasoning Test

SVRT

category classification

Discovery Of Relations by Analogy

DORA

DORA model

Symbolic-connectionist model of relation learning and visual reasoning

A symbolic-connectionist model of relation learning and visual reasoning

Thesis or Dissertation

Doctoral

PhD Doctor of Philosophy