Indicators of mathematics and science education progress are important to policymakers and the public in order to evaluate the quality of education. Recent reports have evaluated existing indicators and suggested core aspects of mathematics and science education that an indicator system should evaluate. This note discusses two fundamental aspects of the quality of educational indicators: their reliability and validity. The note is divided into five sections. The first section discusses recent concerns for the weaknesses in the academic achievement of science and mathematics students in the United States and the need for availability and quality of indicators to assess that achievement. The second section discusses the concepts of validity and reliability as criteria to gauge the quality of indicators. The third section examines the threats to the validity and reliability of indicators. The fourth section presents approaches for evaluating and validating indicators. The fifth section presents the conclusions of the report. The report recognizes the complex task of building an adequate system of indicators of mathematics and science education. Simplification is cited as essential in order to inform debate and policy, and at the same time conflicting with the goal of adequate description. The report suggests that the issue of simplification can be addressed by choosing measures carefully, evaluating the systems of indicators, and employing multiple measures of important constructs. (Contains over 80 references.) (MDH)