Higher education institutions are becoming increasingly engaged in assessing their programs in order to enhance student learning outcomes. States, accreditation bodies and various organizations are calling upon institutions to buildup their accountability towards student learning. Accordingly, multiple assessment methods (both direct and indirect) are being used to gather program assessment data. The paper outlines the framework that has been undertaken to design and implement a Web-based Senior Exit Exam (WEBSEE). This exam is being used as a major direct assessment instrument for an undergraduate program in Computing Information Systems (CIS). The assessment framework is deemed important to enhance the validity of the exam, making it more appropriate for measuring what it tries to measure. The paper also shares some guidelines that have been used to generate reliable and valid exam questions which are directly mapped to program outcomes. The exam questions have been designed to span across a range of IS knowledge areas, knowledge depths and difficulty levels. Our research also outlines the importance to integrate the exit exam in the credit-bearing curriculum to drive students' learning through motivation. The assessment data collected has uncovered some deficiencies that when addressed will help improve student learning. The collected data will also be used as a reference baseline to historically track achievement of program outcomes over the next semesters. The approach and research methodology presented in this study can be useful for IS departments planning to administer similar kind of locally-developed exit exams in future. (Contains 7 tables and 9 figures.)