Characterising the effect of semantic and perceptual similarity in episodic memory
This thesis investigated the contribution of semantic and perceptual similarity to mnemonic discrimination, defined as the ability to recognize previously encountered events and to discriminate them from similar ones to avoid false recognition. When mnemonic discriminations fails, false recognition of similar events is typically attributed to shared memory representations (gist) typically characterized in terms of meaning. In two experiments with young adults, I investigated the effects of multiple semantic and perceptual relations in a mnemonic discrimination task involving pictures of objects. I hypothesised that semantic meaning could be processed at least at two different levels: at the concept level, semantic similarity captures features shared by different concepts. I derived this continuous measure from the Conceptual Structure Account, a feature-based model of semantic memory. At the item level, rated semantic similarity between a depicted exemplar and people’s internal representation of that concept captures the degree to which an item activates its individual representation. For each item, I also measured indexes of visual form and colour similarity. In all the experiments of the current thesis, the study phase was followed by a recognition memory test including studied items, similar lures, and novel items. In Experiment 1 and 2 of Chapter 2 participants studied single and multiple exemplars for each basic-level concept, respectively. Participants were less likely to recognize studied items with high concept confusability, but also to falsely recognize these lures. Thus, greater emphasis on coarse processing of shared features relative to fine-grained processing of individual concepts weakened the basic-level semantic gist shared between studied items and lures. In contrast, false recognition was higher for those lures judged more similar to people’s internal representation of that concept, suggesting reactivation of the studied concept at test by a different exemplar. False recognition was also more frequent for more visually confusable lures. These results suggest that multiple levels of semantic and perceptual similarity contribute to mnemonic discrimination. In Chapter 3, to understand whether initial processing of semantic and perceptual information at encoding is responsible for later true and false recognition, I ran an fMRI study that combined representational similarity analysis (RSA) with the subsequent memory paradigm. To index semantic and perceptual processing that I found relevant in Chapter 2, I created two semantic models to index coarse-grained taxonomic categories and specific object features, respectively, and two perceptual models defining visual properties, like line orientation and colour attributes. Participants encoded images of objects during fMRI scanning. Both perceptual and semantic models predicted later true memory. The strength of the neural patterns corresponding to low-level visual representations in the early visual and inferotemporal cortex was stronger for items successfully recognized versus forgotten. Similarly, greater alignment of neural patterns with object-specific semantic representations in the inferotemporal cortex also predicted true recognition. However, the strength of the neural patterns reflecting nonspecific taxonomic information was stronger for items later forgotten than remembered in ventral anterior temporal lobe, left inferior frontal gyrus and, preliminary, in the left perirhinal cortex. In contrast, inefficient visual processing in posterior regions was associated with false recognition of similar lures. The results suggest that fine-grained semantic as well as visual analysis contributes to accurate later recognition, while processing visual image detail is critical for avoiding false recognition. To conclude, in Chapter 4 I investigated whether the same semantic and perceptual variables that I found to be significant predictors of young adults’ ability to recognize previously studied items, as well as misrecognize similar lures, have a greater impact on older adults’ performance on the same task. Thus, I focused on age-related differences. Contrary to our predictions, Experiment 1 did not show any age-related differences in true and false recognition, or semantic and perceptual modulations on recognition memory. However, Experiment 2 revealed an increase in true and false recognition for more confusable concepts in the older relative to young group when multiple exemplars of each basic-level concept were presented at study. I suggested this effect to be due to a reduction of processing semantic relations across concepts. This result is consistent with gist-based theories of aging that assume that semantic gist between studied items and lures can promote true and false recognition. Altogether, the studies described in the current thesis provide evidence that multiple semantic and perceptual relations can contribute to true and false recognition of objects and that these effects are in part mediated by operation that occur during the encoding phase.