Understanding factors promoting or preventing participants' completion of a massive open online course (MOOC) is an important research topic, as attrition rates remain high for this environment. Motivation and digital skills have been identified as aspects promoting student engagement in a MOOC, and they are considered necessary for success. However, evaluation of these factors has often relied on tools for which the psychometric properties have not been explored; this suggests that researchers may be working with potentially inaccurate information for judging participants' profiles. Through a set of analyses (t-test, exploratory factor analysis, correlation), this study explores the relationship between information collected by administering valid and reliable pre and post instruments to measure traits of MOOC attendees. The findings from this study support previously reported outcomes concerning the strong relationships among motivation, previous knowledge, and perceived satisfaction factors for MOOC completers. Moreover, this study provides evidence of the feasibility of developing valid assessments for evaluation purposes.