This study investigated and compared the effects of two test formats (free response and multiple choice) on English-as-a-Second-Language (ESL) learners' reading comprehension. The tests, together with a checklist of test-taking strategies and retrospective questionnaires concerning more general reading strategies, were administered to 57 ESL graduate students. Two students were also given think-aloud interviews on the tests. Results indicate that the two tests, with identical content but different formats, may not yield measures of the same trait. This was further evidenced by the frequency with which students selected strategies from the checklist to describe their ways of processing the same items in the two formats. A double-check on the checklist's validity showed it to be reflective of the students' strategies. Implications for test designers, teachers creating their own tests, and test validators are discussed. Appended are the test instruments and results. (Contains 41 references.) (Author/MSE)