Regression, Regularization, and Redundancy: Humans' Response to Redundant Inputs in a Linear System

Abstract

In this study, I explored the affect redundant or highly intercorrelated input features had on human participants' ability to learn a linear regression-type task. Earlier studies suggest that, paradoxically, people perform worse with redundant input, something which could possibly be explaining by using regularization to sacrifice training set accuracy for model generalizability. I introduce a novel paradigm for having humans perform linear regression, for calculating what ? weights they learned, and for establishing whether they favored the non-sparse L2 or the sparse L1 regularizer. I found that people form into two distinct groups, on favoring a sparse strategy and the other favoring a non-sparse strategy, but was not able to manipulate which strategy participants adopted. Discussion included implications for psychological and machine learning research.