Purpose: To evaluate pluralistic Extension and Advisory Services (EAS) systems performance and outcomes, and share the experiences made with applying a participatory semi-quantitative approach allowing for cross-country comparability. Design/methodology/approach: The Food and Agriculture Organization of the United Nations (FAO) developed the 'Extension and Advisory Services (EAS) System -- Yardstick' (EAS-Y), a semi-quantitative assessment approach relying on expert-based scores to evaluate the EAS system performance on the one hand, and users' scores to measure the system outcomes on the other. The tool was applied in three countries, Costa Rica, Ecuador, and Peru. Findings: Results revealed an overall weak performance on most assessed criteria. Experts pointed out a lack of adequate policies addressing agricultural extension, insufficient funding, and poor infrastructure. On the other hand, the increased focus on sustainability, increased inclusiveness levels, and steady uptake of digitalization technologies are areas where progress was recently made. On the outcomes side, users perceived EAS contributed mainly to acquiring technical skills, while less to entrepreneurial and social skills. Practical Implications: EAS-Y represented a user-friendly and cost-effective solution to identify performance gaps and assess outcomes in a semi-quantitative way. Therefore, we consider the latter has the potential to be applied to prioritize areas for intervention and guide decision-making processes. Theoretical implications: The commonly existing data gap not allowing for a quantitative evaluation of pluralistic EAS systems can be overcome using a participatory evaluation tool that relies on expert and user's judgments. Originality/Value: We used an innovative evaluation approach to assess pluralistic extension systems in three Latin American countries.