This paper introduces the new Massachusetts Performance Assessment for Leaders (PAL) and uses critical policy analysis to re-examine the validity evidence (using the 2014 Standards for Educational and Psychological Testing and a theory of multicultural validity) for the use and interpretation of the PAL in regards to emerging school leadership. Data sources include two years of PAL test documentation plus candidate surveys and interviews with program directors. The author's role as a test user, faculty instructor, and certified test scorer afforded access to student work, student communications, scorer network training, and state department of education communications and meetings. The paper challenges the content validity, raises questions in regards to evidence based on response processes, internal structure, relation to other variables, consequences, and multicultural validity particularly when the PAL is used as a stand-alone, high-stakes licensure test and offers suggestions to improve the test as a formative assessment.