Human information processing is often modeled as costless Bayesian inference. However, research in psychology shows that attention is a computationally costly and potentially limited resource. We thus study Bayesian agents for whom computing posterior beliefs is costly, such agents face a tradeoffs between economizing on attention costs and having more accurate beliefs. We show that even small processing costs can lead to significant departures from the standard costless processing model. There exist situations in which beliefs can cycle persistently and never converge. In addition, when updating is costly, agents are more sensitive to signals about rare events than to signals about common events. Thus, these individuals can permanently overestimate the likelihood of rare events. There is a commonly held assumption in economics that individuals will converge to correct beliefs/optimal behavior given sufficient experience. Our results contribute to a growing literature in psychology, neuroscience, and behavioral economics suggesting that this assumption is both theoretically and empirically fragile.