The extensive use of simulation-based serious games (SSGs) has made a revolution in educational techniques. As a potentially significant feature for SSG design and evaluation, the term fidelity (the similarity between an SSG and its real reference) emerges and attracts increasing attention. The study of fidelity not only benefits the design, development, and analysis of an SSG with the consideration of improving the learning effect but also contributes to the investment reduction of an SSG. However, the term fidelity is used inconsistently in the current literature. The introduction of new technologies (e.g., virtual reality) and the blend of multiform SSGs also facilitate the extension of fidelity with new connotations. All lead to confusing concepts and vague measure metrics. Besides, the relationship between fidelity and learning effect is still uncertain. A new vision and a comprehensive conceptual framework of fidelity for more general applications are in need. In this paper, further exploration and discussion of these issues in relation to fidelity of SSGs are presented through a systematic review. A general conceptual framework considering both aspects of the SSG system itself and the learners is developed and applied to analyze fidelity in SSGs. Based on that, a discussion on fidelity-related issues of SSG design and development is presented.