Existing methods used to validate self-report questionnaires in foreign language teaching effectiveness have relied on Classical Test Theory (CTT). However, the use of CTT approaches limits the reliability and validity of self-report instruments. The Rasch Model, which is based on the principles of objective measurement, addresses some of the major limitations of CTT-based validation and provides a way to construct linear item and person measures. Thus, the Rasch Model was adopted in the current study as a part of a multi-phase methodological approach to validate the teaching effectiveness questionnaire used in the STARTALK-CHELER Chinese Leadership program. The multiple aspects of validity of the self-report questionnaire were investigated for making an evaluative judgment about the quality of the measure. The multi-phase strategy utilized qualitative, measurement, and statistical techniques. Phase I, content validity, was examined in aspects of relevance, representative, translation, and technical quality. The results showed that some methodological issues, such as content representation gaps, imprecisely translated Mandarin Chinese items, and misfitting items, limited the content validity. The examination of the construct validity regarding the substantive, structural, generalizability, responsiveness and interpretability aspects in Phase II also showed many of the methodological issues, such as person misfit, ceiling effect, and differential item functioning, limited the substantive, generalizability, and interpretability validity. However, the items on the questionnaire were confirmed to measure a single latent construct -- foreign language teaching effectiveness in terms of instructional skills domain summarized in the literature. In Phase III, the results of the independent t-test showed that there were no significant differences between the certified and non-certified teachers on the teaching effectiveness. Finally, a proposed questionnaire in both English and simplified Mandarin Chinese was provided in Phase IV. Further studies should validate the current multi-phase methodology with other foreign language teaching effectiveness assessments. Also, the proposed English and Chinese language versions of the modified questionnaire should be validated to establish their psychometric properties. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]