ERIC Number: EJ1361215
Record Type: Journal
Publication Date: 2022-Aug
Pages: 8
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1092-4388
EISSN: EISSN-1558-9102
Validation of an Automated Procedure for Calculating Core Lexicon from Transcripts
Dalton, Sarah Grace; Stark, Brielle C.; Fromm, Davida; Apple, Kristen; MacWhinney, Brian; Rensch, Amanda; Rowedder, Madyson
Journal of Speech, Language, and Hearing Research, v65 n8 p2996-3003 Aug 2022
Purpose: The aim of this study was to advance the use of structured, monologic discourse analysis by validating an automated scoring procedure for core lexicon (CoreLex) using transcripts. Method: Forty-nine transcripts from persons with aphasia and 48 transcripts from persons with no brain injury were retrieved from the AphasiaBank database. Five structured monologic discourse tasks were scored manually by trained scorers and via automation using a newly developed CLAN command based upon previously published lists for CoreLex. Point-to-point (or word-by-word) accuracy and reliability of the two methods were calculated. Scoring discrepancies were examined to identify errors. Time estimates for each method were calculated to determine if automated scoring improved efficiency. Results: Intraclass correlation coefficients for the tasks ranged from 0.998 to 0.978, indicating excellent intermethod reliability. Automated scoring using CLAN represented a significant time savings for an experienced CLAN user and for inexperienced CLAN users following step-by-step instructions. Conclusions: Automated scoring of CoreLex is a valid and reliable alternative to the current gold standard of manually scoring CoreLex from transcribed monologic discourse samples. The downstream time saving of this automated analysis may allow for more efficient and broader utilization of this discourse measure in aphasia research. To further encourage the use of this method, go to https://aphasia.talkbank.org/discourse/CoreLexicon/ for materials and the step-by-step instructions utilized in this project.
Descriptors: Validity, Discourse Analysis, Databases, Scoring, Aphasia, Transcripts (Written Records), Correlation, Computer Assisted Testing, Vocabulary, Brain, Injuries, Evaluators, Accuracy, Reliability, Error Patterns, Evaluation Methods, Participant Characteristics, Graduate Students, Speech Language Pathology, Check Lists, Comparative Analysis
American Speech-Language-Hearing Association. 2200 Research Blvd #250, Rockville, MD 20850. Tel: 301-296-5700; Fax: 301-296-8580; e-mail: slhr@asha.org; Web site: http://jslhr.pubs.asha.org
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: National Institute on Deafness and Other Communication Disorders (NIDCD) (DHHS/NIH)
Authoring Institution: N/A
Grant or Contract Numbers: 3R01DC008524