The assessment of oral proficiency in English for Specific Purposes (ESP) contexts remains a central challenge for researchers, test developers, and classroom teachers (Knoch & Macqueen, 2020). This problem is apparent especially in Tourism and Hotel Management (THM) programs, where students are tested mostly based on grammar, vocabulary, and listening through multiple-choice items because of several logistical issues such as crowded classrooms and resource intensive nature of administering an oral communication assessment (Hsu, 2014). One way to solve this problem is to use authentic role-play tasks that can predict the target language performance of a test-taker in a simulated context (Douglas, 2000; Manias & McNamara, 2016) by using spoken dialog systems (SDSs) in which computers act as conversational agents. However, there have been limited applications of SDSs for assessing oral communication skills in ESP assessment so far. To fill this gap, the current dissertation study aimed to develop a prototype of SDS-mediated Tourism English oral communication assessment and investigate the level of support for the first four inferences (domain description, evaluation, generalization, and explanation) in the interpretation/use argument for the computer-based Tourism English oral communication assessment (CTEA). Based on evidence-centered design (ECD) model (Mislevy, Steinberg, &, Almond, 2003) as the guiding framework for test development, three counter service scenarios were designed and developed using a specialized SDS in which computer acts as a hotel guest and examinees act as a receptionist. A total of 30 EFL students majoring in a THM program at a Turkish state university, 12 domain experts, and four raters participated in the study. Employing a mixed-method multiphase research design (Creswell & Plano Clark, 2012), the study collected various types of data such as students' CTEA test scores, their Tourism English midterm grammar and vocabulary scores, Tourism English course instructors' evaluations of the students, stimulated recalls, individual semi-structured interviews,Ho and responses on post-test surveys and post rating surveys. The four inferences were well supported by the evidence collected from the data analysis. The domain description inference was supported by domain experts' overall positive opinions about the authenticity and effectiveness of the CTEA tasks in eliciting oral discourse from test-takers. The evaluation inference was backed by the findings obtained from the students' interview and survey data that suggested an overall positive perception of the CTEA. The students also thought that the CTEA allowed them to demonstrate their oral communication skills in simulated hospitality-related situations and underlined a potentially positive impact on their learning processes in a future CTEA integration into the Tourism English assessment. The evaluation inference was also supported by the additional findings obtained from all four raters who reported that the SDS was consistent across all test-takers and thought that the SDS could potentially eliminate the effect of partner variability on their performance. The generalization inference was backed by the findings of the G-theory analysis on 360 composite scores assigned to 30 students' CTEA performances on three different task prompts. G-study findings revealed that the score variance was 66.5% for person and there was minimal rater severity, meaning that two-thirds of the score variance was due to the test-takers' Tourism English oral communication ability. Additionally, D-study findings yielded a substantially high dependability index ([phi] = 0.90) for the current test design and an acceptable level of dependability ([phi] [greater than or equal to] 0.70) was obtained in all testing scenarios, except for one rater and one task scenario. However, the assumption for the appropriateness of test task specifications for producing parallel tasks was partially supported because of the technical difficulty and resource intensive nature of developing parallel SDS-based test tasks. Finally, the explanation inference was supported by the results of the concurrent correlational study between the CTEA scores and external measures (grammar scores, r=0.62; vocabulary scores, r=0.67; and teacher evaluations, r=0.75). This inference was also supported by the stimulated recall data findings that pointed to an overall potential of CTEA in providing opportunities for test-takers to demonstrate a variety of strategies to complete the tasks. The study contributes to knowledge about practical assessment of ESP, specifically Tourism English, oral communication at universities by presenting significant implications on the design and use of an SDS-based role-play tasks in ESP assessment, as well as learning and teaching processes. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]