Concerns surrounding subscore use, which have traditionally been examined in large-scale educational testing programs (e.g., SAT, GRE)--namely, that "information obtained from subscores that seems intuitively useful may not contain additional information over and above the total score" (p. 2)--have been argued to apply equally to assessment and evaluation in higher education settings. One widely adopted survey which uses subscores (referred to as "engagement indicators") is the National Survey of Student Engagement, or NSSE. Thus, the purpose of the current study was to evaluate whether ten subscores reported from the NSSE survey demonstrated sufficient reliability and distinctness to support validity of interaction and use at the individual student and aggregate college levels. Survey responses from a sample of three fourth-year student cohorts at a large, land-grant institution were used to evaluate subscore utility. In particular, analytical methods grounded in classical test theory (CTT) and generalizability (G) theory were employed. Haberman's (2008) method, which is grounded in CTT, addressed whether each of the ten reported NSSE engagement indicators added value above and beyond reporting a total observed score for student engagement. Subsequently, Jiang and Raymond's (2018) multivariate generalizability method examined whether the collective profile of observed NSSE scores provided sufficient overall score profile utility. Following NSSE's standard reporting scheme, both methods were applied to individual student level subscores and aggregate college level subscores. The results of this study suggested that the psychometric qualities and diagnostic value of NSSE's primary reporting structure, the engagement indicators, varied based on the unit of analysis. First, the current study established via Haberman's (2008) method that the engagement indicator subscores consistently demonstrated added value above and beyond reporting a total score. However, the aggregate reliability indices obtained from Haberman's (2008) method and results of the generalizability theory method raised concerns about the psychometrical quality of the subscores at the aggregate college level. Ultimately, when evaluated fully in their aggregate form--the level at which NSSE scores are generally reported--the engagement indicator subscores were not sufficiently reliable nor substantively meaningful to warrant use for institutional decision-making. Ultimately, these findings presented implications for both research and practice. While the NSSE engagement indicator subscores were able to reliably capture individual level scores, serious caution should be exercised when reporting engagement indicator subscores at the college level. Rather than capturing meaningful differences between colleges, the aggregate engagement indicator subscores merely reflected the many individual differences across all students institution-wide. Relying on those aggregate measures to inform institutional decision-making at the college level would be ill-advised within the context of the focal institution. Thus, the significance of this study lies in its ability to inform decision-making for the hundreds of colleges and universities currently relying on NSSE to inform institutional decision-making. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]