Most educational researchers have heard the adage, "If you want to measure change in performance, don't change the measure." At the same time, however, what students need to know and be able to do may change over time as research on teaching and learning provides new insights into the educational process, science and technology advance, and expectations for student achievement rise in response to global competition. Assessments must reflect these various changes or risk becoming irrelevant. But at what point do changes in an assessment mean that it is measuring something fundamentally different than the old version, so that the results generated by it can no longer be validly compared to the results generated by previous versions of the assessment? The focus of this paper is to address this question as it pertains to reporting the results of NAEP assessments that are changed to reflect new or revised assessment frameworks. The purpose of this paper is to recommend guiding principles, suggested studies, and decision-making processes to NCES leadership to assist them in determining whether the results generated by an assessment based on a new NAEP framework can be validly reported on the same trend line as previous versions of the assessment. The authors begin with a discussion of principles that NCES could follow in making decisions about whether or not to break an existing trend line. This is followed by suggestions for the types of studies and decision-making procedures that might be found useful for such deliberations. A summary of changes to NAEP assessment frameworks and trends is appended.