Drawing on the combined theoretical lenses of positioning theory and academic literacies, this article presents case studies of four children from one urban classroom, two of whom scored at or above proficient on the large-scale writing assessments required by their district and state and two of whom scored below. Using criteria from state rubrics, we closely analyzed the writing products children produced for high-stakes assessments and classroom writing projects as well as drew on a range of qualitative data to contextualize children's writing within the complex relationships with writing observed across the school year. Our findings suggest test scores may be inaccurate or highly malleable based on relations between the features of the writing children produced, students' identities as writers and preferred practices, quirks of the testing context, and arbitrary features of the test itself. Indeed, our analyses found that some children's test scores misrepresented their capabilities as demonstrated in the writing they produced both within and outside of the testing situation. Furthermore, the form of the assessments risked positioning these children in just the ways that would frustrate rather than promote their attempts to put their best writing on the page. Our data suggest that the children's test scores did not provide the information about achievement in writing that such tests are assumed to convey and that both the form of on-demand writing assessments and the dichotomized sorting they facilitate potentially undermine some of the very goals, often articulated by policy makers, underlying the push for accountability through testing. (Contains 7 figures.)