Direct measurement of learning outcomes in higher education: A proposal of nine standardized scales for continuous improvement in engineering programs

Despite the growing interest in measuring learning outcomes as evidence of comprehensive education in higher education institutions, objectively assessing these outcomes remains a challenge. This study addresses this problem by developing and validating nine standardized direct assessment scales tailored for engineering programs, with the goal of enhancing quality assurance. Study 1 focused on designing indicators for the scales through expert workshops and peer judgment processes, looking for content validity using Fleiss Kappa. Study 2 involved instructors using these instruments to assess various learning outcomes in a STEM-focused university. Analysis of 1420 assessments revealed strong validity and reliability indicators, including unifactorial structures, high reliability, discrimination, and appropriate difficulty levels for all scales. The research contributes evidence supporting the content validity of these assessment instruments, offering valuable tools for researchers and practitioners in engineering education to support continuous improvement. These scales have the potential to support future comparative research, inform quality assurance decisions, and enhance comprehensive education practices. Finally, the study discusses its limitations and provides directions for future research.