Test validity can be jeopardized to the extent that test scores are contaminated by factors other than those intended by the test. Many method specific factors and examinees' personal characteristics can influence test performance. In this study the contribution of four construct-irrelevant factors, namely, exam anxiety, reading anxiety, ambiguity tolerance, and attitudes towards reading to reading comprehension test performance was examined. Correlation and multiple regression analysis showed that reading anxiety and attitudes toward reading significantly contribute to reading test performance while exam anxiety and ambiguity tolerance do not. This finding shows that examinees' reading anxiety and attitudes toward reading can invalidate the reading comprehension test scores to a certain degree. Implications and limitations of the findings are discussed and some recommendations for test administrators and teachers are provided.