Recently most sub-Saharan countries have successfully broadened access to basic education. However, empirical and educational reports from this region have consistently documented children not acquiring the foundational literacy skills. This study examined the differences in the level of literacy acquisition between pupils from urban and rural backgrounds in Tanzania. A mixed-method research approach under the concurrent mixed design was employed. A total of 200 early grade children, 120 parents, 20 teachers were recruited. The data were collected by the Early Grades Reading Assessment, semi-structured interviews, parent questionnaires, and documentary analyses. Findings indicated that urban children outperformed rural children when it came to every literacy aspect tested, except reading comprehension. In both areas, girls outperformed boys, although rural boys were generally over-aged. The home learning environments for children from rural areas were found to be limited with less support than even for those from the poor urban areas. Regardless of urbanicity, poor and extremely limited teaching and learning facilities, large class sizes, and curriculum issues were pointed out as the main hindrances for children to acquiring literacy skills. These findings have implications for policymakers, teachers, parents, and other stakeholders in this region calling for joint reform efforts to improve early literacy acquisition.