Research in the context of decentralization has primarily focused on the role of school-based actors in improving educational outcomes for students in low- and middle-income countries (LMICs). The role of the "middle tier," such as subnational staff at the regional or district levels of educational leadership within the education bureaucracy, has not been systematically understood. We conduct a rigorous review of academic and gray literature in the English language and find that (1) research on the middle tier is limited in the discourse on education reform in LMICs; (2) in the available research, there is tension in the theoretical conceptualization and the practices of the middle tier between different fields of study and research methods; (3) we know even less about whether and how the middle tier supports improvements in student learning. Our article calls on researchers to investigate how the middle tier can strengthen education service delivery in LMICs.