A study investigated the effect of test item type (multiple-choice or open-ended) on performance reading comprehension tests given in both the student's native language and a second language. Subjects were 24 native Arabic-speaking and 38 native Hebrew-speaking students at Haifa University (Israel), all enrolled in a course in English as a second language. English language texts were selected from an Israeli standardized test, with two test item versions: multiple-choice and open-ended. Texts in Arabic and Hebrew were drawn from practice books designed to prepare students for a psychometric test, and similarly, two types of test item were prepared for each. All texts were controlled for readability, length, and neutrality of topic. For each test, reading processes were examined using the paraphrase/translation segment of a think-aloud protocol. Results across test context (test items vs. paraphrase/translation), across language type (English as a second language vs. native language), and native language group (Arabic vs. Hebrew) are examined, and implications concerning the construct validity of reading comprehension tests are discussed. Contains 19 references. (MSE)