We report on research conducted during a language assessment literacy (LAL) project set up for employees at a university language centre. Teachers, students and an external accreditation body requested a revision of the end-of-course exams to increase comparability, transparency and alignment to course aims. We used a collaborative approach where stakeholders involved in test development, quality control, use and interpretation brought their experiences, skills, knowledge and needs to the table, following Taylor's (2013) view of differentiated LAL needs for different stakeholders. The project involves teachers (varying in numbers), five course/assessment coordinators and three researchers with a background in assessment (principal researcher, two research assistants, i.e. the authors). LAL training is needs-based, participant-driven, and evaluated in regular intervals. In response to these participant evaluations, delivery, content and focus of the training are adjusted. We present insights from five evaluation points gained during the first 3 years. We found that the collaborative and responsive LAL project was perceived as useful and effective by the participants, leading to learning gains and positive changes in assessment, teaching and coordination practices. Our insights contribute to a better understanding of the conditions that are conducive to designing and delivering effective LAL development projects.