The orchestration of collaborative learning activities in technology-enhanced classrooms has become a nontrivial endeavor for educators. Depending on the behaviors and needs of students that emerge in real educational situations, educators may need to orchestrate activity adaptations on the fly. These adaptations may range from the provision of additional scaffolding by the educator (e.g., the educator's participation in a group discussion) to a change in the planned pedagogical scenario (e.g., the duration). This article aims to contribute to the orchestration of technology-mediated collaborative learning sessions in a classroom context. We present the design, implementation, and evaluation of a teacher-facing dashboard that supports teachers in orchestrating scripted collaboration. Evaluation studies were conducted in 16 classroom sessions. The findings indicate that teachers found the information on the dashboard to be actionable and help facilitate just in time support to student groups.