Few juvenile justice group experimental and quasi-experimental literacy-focused studies document fidelity of implementation (FOI). Even fewer of those studies consider the psychometric properties of the FOI instrumentation used. The aim of this study was to examine the psychometric properties of a brief FOI instrument used with English language arts classes in a juvenile justice school as part of an Institute of Education Science research project. Six teachers were observed a total of 209 occasions across three instructional conditions (Business-as-Usual, REACH, REACH + Strategic Instruction Model) within one long-term juvenile justice facility with approximately 300 male youth aged 12 to 18. Findings indicated the instrument has high internal consistency at 0.98. Rasch modeling suggested the instrument measured two separate constructs (e.g., instruction and behavior management). Developmental pathway analyses indicated items are arranged on an ability continuum with some items being easier or harder for teachers. Findings suggest the FOI instrument has the potential to be used in English language arts classes within a juvenile justice facility. Future directions include developing additional easier and harder items to account for the abilities of less and more skilled teachers. Moreover, research is needed to examine the FOI instrumentation usability and feasibility across similar and different types of juvenile justice facilities.