Behaviorally anchored rating scales (BARS) are an essential component of structured interviews. Use of BARS to evaluate interviewees' performance is associated with greater predictive validity and reliability and less bias. BARS are time-consuming and expensive to construct, however. This report explores the feasibility of gathering participants' responses to structured interview questions through an online crowdsourcing platform and using those responses to develop BARS. We describe the development of 12 structured interview questions to assess four applied social skills, elicitation of responses to these questions in the form of critical incidents from 68 respondents, and the creation of BARS from these critical incidents. Results indicate online participants are able to produce responses of sufficient quality to generate BARS for evaluating structured interview performance. We conclude by discussing limitations to this approach and future directions for research and practice.