Research has repeatedly shown self-efficacy to be associated with course outcomes in CS and across other fields. CS education research has documented this and has developed CS-specific self-efficacy measurement instruments, but to date there have been only a few studies examining interventions intended to improve students' self-efficacy in CS, and several types of self-efficacy interventions suggested by previous research remain to be tested in CS. This study attempts to address this lack of research by reporting on the results of a trial intervention intended to improve students' self-efficacy in an introductory programming course. Students were recruited to complete a self-evaluation task, which previous research has suggested could have a beneficial impact on self-efficacy, which should in turn have a beneficial impact on course performance. Participating students' course outcomes and self-efficacy were compared with those of the students who did not complete the self-evaluation task, using propensity score weighting adjustments to control for differences between the groups on entering characteristics and prior values of self-efficacy and course outcomes. We found that, whereas there was only marginal evidence for the self-evaluation intervention having a direct effect on self-efficacy, students who completed the self-evaluation task had significantly higher project scores during the weeks they were asked to complete it, compared to the students who did not participate. These findings suggest that there are potential benefits to incorporating self-evaluation tasks into introductory CS courses, although perhaps not by virtue of directly influencing self-efficacy.