The Investigators studied scoring reliability of Candidates' ten day unit plans of instruction through prescribed action research projects, across three academic years. Scoring of the projects in year one provided opportunities for further refinement of the action research evaluation methods in year two. Across three terms in years one and two scoring reliability was realized. However, although reliability was realized in term two of year two, new scoring issues arose. Subsequently in year three of the study reliability was not realized across rater and interraters due to a lack of updating points earned on projects in mastery learning, lack of abiding by the rubric for scoring, and informal discussions between the rater and some interrators affecting scoring of projects. Discussion includes changes that likely led to reaching early satisfactory reliability levels, as well as reasons for not realizing reliability in the most recent year. Recommendations follow that may be helpful to education Units seeking to learn more about reliability issues when scoring Candidate items. (Contains 7 tables.)