Adventure sports coaches must make sound judgements and effective decisions to ensure they can manage the significant risks present in an operating environment that is hyper-dynamic and has significant environmental complexity (Collins & Collins, 2013). High-level stakeholders in adventure sports require coaches demonstrate competency to fulfill their responsibilities to look after their clients. Most adventure sports qualifications identify the importance of decision making, however, by using competence-oriented syllabi they may not provide effective methods to evaluate a prospective adventure sport coach's decision making. This paper identifies the need for expertise-oriented evaluation of decision making and explores decision support tools and educational validity theories in the evaluation process. It also considers how Collins and Collins' (2021) Big 5 style approach represents an exemplar decision support tool and may provide educational assessment validity, so presenting a sound methodology for coach educators to evaluate the quality of an adventure sports coaches' decision making.