A large number of students across the globe each year enroll in general chemistry courses as an academic requirement to obtain their degree. Although many take chemistry courses, it is not a subject sought out by many as a potential career. In some instances, chemistry hinders students from achieving their career goals. A plethora of chemical education research has focused on improving student attitude, self-efficacy, and motivation to enhance academic performance and retention in chemistry. However, only a few reports focus on the factors that affect student perception and self-efficacy towards chemistry. These factors are important as they can help us implement targeted interventions to improve perceptions and self-efficacy as we seek to increase diversity in STEM fields. In this research study, the most influential factors that affect a student's perception of chemistry are uncovered, and whether these factors are related to gender identity, letter grade, or pursuit of chemistry as a career. For our study population, the "course instructor" and "course structure" are the two most influential factors in a student's perception of chemistry. In addition, academically low-achieving students (i.e., students who earned Cs or lower in a course) are more likely to list the "course structure" as an influential factor, and high-achieving students (i.e., students who earned Bs or higher in a course) are more likely to select the "course instructor" as an influential factor. The majority (66%) of students who selected the "course instructor" as an influential factor believed that they would perform well in future chemistry courses, while 47% of those who selected the "course structure" had the same belief in their future chemistry performance. Overall, less than 11% of the study population (51 of 447 students) were interested in pursuing chemistry as a career after completing CHEM 1. However, the answer to increasing the number of chemistry majors could be held within course design and teaching pedagogy. This research study seeks to highlight the relationship between gender and letter grade with factors that influence perception of chemistry, and we hope the results can guide instructors as they consider course structure and teaching pedagogy.