In the spring of 1988, the Massachusetts Department of Education administered its second biennial assessment (the Massachusetts Educational Assessment Program), testing students in grades 4, 8, and 12 in reading, mathematics, science, and social studies. This report discusses the use of open-ended questions in one form of the science tests. One-twelfth of the fourth graders, one-sixteenth of the eighth graders, and one-twentieth of 12th graders who were tested in 1988 received a test form that contained some open-ended questions. These open-ended questions were designed to examine two areas of science comprehension: scientific inquiry, and the understanding of scientific concepts. Sample test items and an analysis of student responses are provided for each grade level in the two areas of science understanding. Although students appeared to know and recognize the rules and principles of scientific inquiry when presented as stated options, they seemed to be baffled by unstructured situations that demanded an application of these principles. The examination of students' conceptions of some basic scientific theorems revealed many inadequacies in students' abilities to apply the scientific method to real problems. Student responses underlined the fact that many assumptions made about what students know and understand are unfounded. Open-ended questioning appears to be a way for teachers to really understand how their students rationalize the world around them. Student responses and their frequencies are summarized in an appendix. (SLD)