This paper furthers the understanding of literacy being developed through this Special Issue by bringing a pluralistic, complex account of literacy into conversation with assessment. The rise of neoliberalism in education--and its related focus on standards, accountability, and efficiency--has seen literacy positioned as the "core business" of contemporary education systems and, accordingly, a construct that must be rendered readily measurable, benchmarked, and accounted for. Through a critique of current literacy assessment practices in Australia relying on standardized testing to achieve such accountability using National Assessment Program for Literacy and Numeracy and Literacy and Numeracy Test for Initial Teacher Education as examples, we argue that the resulting dominant account of literacy is both reductive and deficient. On the one hand, we show how standardized literacy testing fails to recognize students' actual potential, in terms of the diverse and varied plurilingual and cultural meaning-making resources that many students now bring to literacy, beyond using Standard Australian English alone (and as a monolingual speaker would use SAE as an isolated language system). On the other, we show how standardized tests can render the assessment of literacy as effectively meaningless, given difficulties discriminating between test-takers' full range of literacy skills in meaningful ways. We conclude by considering alternative practices with potential to more accurately assess learners' literacy capabilities from a plurilingual perspective, and implications for education policy.