Multiple meta-assessment measures of a quality process: toward institutional effectiveness

• Published in: Quality assurance in education 2020-04, Vol.28 (2), p.123-136
• Main Author: Pham, Nhung Thi Tuyet
• Format: Article
• Language: English
• Subjects: Academic Achievement; Academic Education; Academic Standards; Accountability; Accreditation; Accreditation (Institutions); Budgeting; Case studies; College Administration; Curricula; Decision Making; Educational Quality; Evaluation Methods; Evidence; Feedback (Response); Institutional Evaluation; Interrater Reliability; Intervention; Learning; Learning Processes; Management Systems; Organizational Effectiveness; Peer Evaluation; Performance evaluation; Politics of Education; Pretests Posttests; Program Evaluation; Psychometrics; Quality; Quality Assurance; Quality Control; Resource Allocation; Self evaluation; Self Evaluation (Individuals); Stakeholders; Strategic Planning; Student Improvement; Students; Summative Evaluation; Teaching Methods; Total Quality Management; Transparency; Universities; Web Sites
• ISSN: 0968-4883; 1758-7662
• DOI: 10.1108/QAE-08-2019-0080

Purpose The purpose of this study is to share quality process experience from a US comprehensive university to use both direct (participation rate and assessment quality) and indirect assessment measures (assessment survey) to evaluate the quality process. Design/methodology/approach A mixed method design was used to evaluate the quality process. The qualitative design used thematic analysis to find the common themes from the assessment survey. The quantitative design included reliability and internal consistency for the survey and inter-rater reliability for the peer review. Findings Findings indicated that participation rate and assessment quality improved over the years. Faculty provided positive feedback about assessment website, the peer review process and feedback but negative perspectives on the assessment management software and heavy workload on assessment activities. The researcher shared the actions made based on three-year assessment results, especially how leaders have used it to align with planning and budgeting to close the institutional effectiveness loop. Research limitations/implications This research had two limitations. The inter-rater reliability of the peer review was not high and the number of peer review reports in two years were so different, so the results could not be checked for statistical significance. Practical implications The meta-assessment results could have important implications for other universities to improve the quality process. Originality/value There have been limited studies on using both direct and indirect assessment measures to evaluate the quality process. This study uses multiple measures to provide accurate results for administrators’ decision-making in resource allocation.