The Effects of Color in Recognition of Images in Multiple-Choice Displays By People with and Without Aphasia

Abstract

Language assessments for people with aphasia often rely on the use of images. Images are presented together with a verbal stimulus and participants are asked to choose the image that corresponds to the verbal stimulus. It is assumed that if a person chooses an incorrect image, he or she has language comprehension deficits. However, other non-linguistic factors can influence image recognition processes, such as stimulus characteristics and verbal or motoric response requirements associated with target selection. Color has been shown to facilitate image recognition in language-normal individuals and in people with aphasia. However, traditional tasks to assess the influence of color on image recognition rely on verbal responses, which pose serious response confounds in individuals with aphasia. This study utilized eye-tracking to capture individuals’ responses, avoiding response confounds associated with traditional assessment methods. The overall goal of the present study was to determine the role of color in multiple-choice image displays on language-mediated eye movements in individuals with and without aphasia. Specifically, it was determined if people with and without aphasia would recognize color images more easily compared to black-and-white line drawing images that correspond to a verbal stimulus in multiple-choice image displays. It was also determined if individuals with aphasia would fixate longer on images that share the same conceptual color as the verbal stimulus in color and black-and white images. A group of ten language-normal participants and five participants with aphasia viewed 40 multiple-choice image displays containing color images and black-and white images which were presented together with a verbal stimulus under two conditions. In the target condition, the verbal stimulus corresponded to one of the images in the display. In the competitor condition, the target image was replaced with a color competitor image while the same single-word verbal stimulus was presented as before. Eye movements were recorded as individuals looked at a computer screen and listened to words. The eye-movement measures proportion of fixation duration on the target image (PFDT) and first pass gaze duration on the images (FPGD) served as the dependent measures. A pointing version of the task served as a control measure and validation that individuals indeed understood the verbal stimulus and identified the image correctly. Surprisingly, FPGD and PFDT of color images were not found to have a significant advantage over FPGD and PFDT allocated to black-and-white images. In fact, FPGD for black-and-white target images was significantly greater than FPGD allocated to color images. No significant group differences were found in the target condition. In the competitor condition, participants fixated disproportionately longer on both color and black-and-white competitor images compared to the other images in the display, but no significant difference was found between the color images and the black-and-white competitor images. A significantly greater disproportionate allocation of fixation duration allocated to competitor images that were related semantically to the verbal stimulus compared to those who were not semantically related was observed. This result highlights the need to carefully control for semantic association between verbal stimuli and competitor image in addition to physical stimulus characteristics. In conclusion, based on the current findings, color did not facilitate image recognition in people with aphasia or control participants and a semantic competitor effect was observed rather than a color competitor effect.