This article presents evaluation data regarding the MAPIS3 architecture which is proposed as a solution for the data-transfer among various tools to promote flexible collaborative learning designs. We describe the problem that this architecture deals with as “tool orchestration” in collaborative learning settings. This term refers to a situation where data relevant to a collaborative learning activity need to be forwarded to and processed by various learning technological tools (e.g. a forum, a pedagogical agent, a service or a software component that provides a specific functionality, etc.), in order for the collaborative activity to be efficiently represented and teachers' pedagogical level decisions implemented. To facilitate data-transfer among the various tools and accomplish flexible interventions during runtime, the architecture employs a key component (“mediator component”) which makes use of an IMS-LD based representation of the activity. By implementing the architecture tradeoff analysis method in three case studies, evaluation data regarding the proposed architecture have been recorded and are presented in this paper. Targeted stakeholders (learners, teachers, and developers) provided valuable insights on the capacity of the architecture to efficiently facilitate tool orchestration during the realization of a flexible IMS-LD based course. Additionally, limitations of IMS-LD are discussed and suggestions are made on how to tackle these constraints and increase flexibility during tool orchestration in collaborative learning deployment.