Rater-mediated activities in educational research occur when an expert judge or rater utilizes an instrument to judge persons or items and generates scale scores. Scale scores are from a subjective judgment and must undergo a quality control measure called rating quality. Rating quality in this study is broadly defined as the extent to which scores are estimated with reliability, are unbiased, and contain little error variance. The purpose of this study was to utilize two frameworks, Classical Test Theory (CTT) and Item Response Theory (IRT) -- Many-Facet Rasch Model (MFRM), and to examine the extent to which they differ in their utility to assess rating quality in standard-item alignment. Results indicated that CTT exercised the ability to assess agreement/reliability but was limited in its utility to detect systematic error in expert alignment scores. However, when experts aligned mathematics items to College and Career Readiness standards, the IRT-MFRM framework demonstrated the capacity and utility to assess reliability and systematic variance. The IRT-MFRM framework described in this study also contains the utility to examine expert rating patterns, item-level statistics, and rating scale quality. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]