Teaching sports geography in higher education on the surface seems like it would be a great course to draw students in, get them exposed to geographic concepts, tools, and techniques, and the perfect forum for using examples to which students can relate. However, there are few instances of a sports geography course being in a regular rotation of courses. A deeper look at why this is reveals some challenges both administratively and academically. This reflection paper discusses some of these issues from the perspective of faculty members in the U.S. higher educational setting as well as some challenges of including physical geography in such a course. Although students tend to understand how geography and sports are intertwined from human geography concepts, physical geography concepts are also an important part of this understanding and do not get a lot of attention during these types of discussions. This reflection does consider the many ways that physical geography needs to be part of academic courses on sports geography.