This paper reports on an evaluation of Net-Detectives, a creative online role play activity aimed at 9-12 year olds. Net-Detectives forms part of Kidsmart, an Internet awareness programme aimed at school children. It was evaluated through a multiple method data collection using questionnaire surveys, follow up telephone interviews with teachers and observation in use in three UK schools during the summer term of 2003. All sources of data collected agreed that a number of benefits arise from participation in the Net-Detectives online role play. The pupils learned about Internet safety in a motivating and challenging environment. They empathised with others gaining an understanding of their motivations and practiced the ICT skills being taught. They particularly enjoyed being "detectives" and interacting with others outside their school. However, the role of the "hosts" in preparing participants and in moderating their communication is key to the success of the online role play. This preparation needs to be thoroughly planned for both the teachers whose classes are participating and for their pupils as uncertainty leads to extra communication that interferes with the smooth running of the role play exercise. It was very clear that first the technical issues have to be resolved and then, issues to do with the learner's comprehension and expectations of the task, before social interaction online can result in learning. The paper concludes by presenting the recommendations suggested by this study for hosting an online role play in the school environment.