The purpose of this paper is to provide states and school districts with processes they can use to help ensure high-quality data collection during teacher observations. Educational Testing Service's (ETS's) goal in writing it is to share the knowledge and expertise they gained: (1) from designing and implementing scoring processes for the Measures of Effective Teaching (MET) project, which used content-neutral and content-specific observation instruments for measuring teacher practice; and (2) from other experiences they have had with observation systems. The authors broadly outline what is involved in setting up such a system, including considerations related to selecting an observation instrument, training observers, certifying observer proficiency, and conducting post-training observations of classroom practice.