This collaborative action research highlights the need for developing students' evaluative competence and self-reflection by embedding self-and-peer assessment into online instruction. Over the course of a semester in an online master program in mathematics and computer sciences, students conducted research on assigned topics, held presentations, formulated meaningful questions for peer-assessment, and finally engaged in Certainty-based Marking (CBM) by rating how certain they are that their answer is correct. The goal of using CBM was to foster students' careful reflection and provide feedback to teachers about students' status of knowledge. A mixed-method approach was used to triangulate data from two sources: (a) assessment artifacts, i.e., student-generated questions and CBM, as evidence of learning, and (b) students' attitude captured through 'Task Perception Questionnaire'. Assessment data were analyzed by three domain experts based on their judgement of 'quality' and Kappa measure was used to assess inter-rater consistency. Quantitative analysis of questionnaire data, coupled with instructors' observation, indicated positive attitudes (engaging and useful) towards CBM among students. We conclude with a discussion of limitations as well as implications of this classroom research project.