Although algebra often functions as a gatekeeper to higher-level mathematics courses and higher education admissions, few quality measures exist for assessing conceptual understanding. This study explored the technical adequacy of three procedural and two conceptual algebra screening measures. We administered three rounds of assessments throughout an algebra course to 2,021 secondary students of 31 teachers in three states. Assessments included procedural and conceptual screening measures and additional criterion measures: teacher ratings of students' algebra proficiency, course grades, results of two project-specific algebra proficiency exams, and state test scores. Descriptive and correlation analyses were used to investigate measure scores, alternate-form and test-retest reliability, concurrent validity, and predictive validity. Procedural measure results indicated high levels of reliability (r = 0.72-0.99), and moderate concurrent and predictive validity (r = 0.36-0.64; 0.36-0.58). The conceptual measures produced moderate to low levels of validity (r = 0.10-0.44). The procedural measure results suggest they may be suitable for use as screening measures, pending further revision and diagnostic testing, while the conceptual measures did not produce acceptable results for current implementation. The findings contributed to measure redesigns to bolster their use as mathematics proficiency assessments with algebra students.