Locally developed occupational tests were administered to 16- and 17-year-olds in a government-sponsored vocational education program in the United Kingdom over a six-month period in 1984. Job skills were tested in two occupational areas: use of a micrometer and invoice completion. Some performance tests were designed by researchers and some by the trainees' supervisors, who administered the tests. Standardized national tests were not used. One purpose of this study was to examine the feasibility and implications of this system of decentralized testing, in which practical tests were designed and administered following central guidelines. The other major purpose of this study was to examine the factors affecting test reliability, within-site examiner reliability, and the comparability of test content and standards across sites. Results indicated that of the examiners who were inconsistent in their application of judgment criteria, five percent were inconsistent in relation to their own micrometer exercises, and 45 percent were inconsistent with their own invoice tests. There appeared to be considerable variation in the standards used between different raters. Results from studying the internal reliability estimates revealed the dangers of a single assessment. The trainees' behavior was extremely variable when performance on two different tasks was assessed. Appendixes describe the methodology more fully and provide formulae for reliability coefficients. Thirteen tables are offered. (GDC)