Students can experience a variety of emotions while completing assessments. Some emotions can get in the way of students performing their best (e.g., anxiety, frustration), whereas other emotions can facilitate student performance (e.g., engagement). Many new, non-traditional assessments, such as automated conversation-based assessments (CBA), are designed with the intention to create a test-taking experience that maximizes beneficial emotions for students and minimizes detrimental emotions. However, there is a paucity of research on students actual emotional experiences during these non-traditional assessments. Across two studies, we investigated students moment-to-moment emotions during two CBAs that differed on construct (science inquiry, mathematical reasoning) and environment (virtual world, chat box). We found a similar set of emotions to frequently occur across the two studies and came to the preliminary conclusion that students have a generally positive experience with CBAs and that boredom, confusion, curiosity, delight, engagement/flow, frustration, happiness/enjoyment, hope, and pride are the prevalent emotions in CBAs. The temporal dynamics of emotions as well as the emotion-performance relationship were also investigated across the two studies. Lastly, we discuss how these findings can inform the development of emotion-sensitive CBAs that can facilitate students being able to perform to the best of their ability.