A system-based approach to expert assessment work presupposes collaboration between the Educational Psychological Service (EPS) and key stakeholders to capture students' needs. Nevertheless, few studies have explored what challenges to collaboration exist and how they can be resolved. The purpose of this study was therefore to fill this knowledge gap by exploring EPS advisers' experiences of challenges in collaborating with key stakeholders, including students, parents, teachers and principals. The results of eight interviews revealed challenges for EPS advisers in capturing the student's voice, gaining legitimacy from teachers, involving principals, and being parents' spokesperson, as well as challenges related to the perceived ambiguity of the EPS's mandate. The findings suggest a lack of collaborative competence to lead a joint process of knowledge development; such competence could prevent conflicting expectations of EPS`s mandate, confusion about roles and contribute to trustful relationships between EPS and key stakeholders. A two-part strategy for a collaborative approach to expert assessment work is suggested.