Theory-based evaluation of the impact of Master’s programmes in development studies: Insights from a mixed-methods and multicultural alumni action research project

• Published in: Evaluation and program planning 2023-04, Vol.97, p.102228, Article 102228
• Main Authors: Holvoet, Nathalie, Casten, Wanda, Demissie, Eshetu Woldeyohannes, Dewachter, Sara, Gamboa, Marian Kaye C., Adhanom, Tewelde Gebremariam, Ibrahim, Abdurahman Hamza, Makundi, Hezron, Manguni, Grachel, Manyuru, Georgina Angela, Mugabi, Nicholas, Nawanda, Yahya E., Nguyen, Chau Hoai, Pallangyo, William Amos, Pascual-Villar, Rose, Saavedra, Luz Prado, Salgado, Mariluz Torres, Tysmans, Nash, Vu, Anh Ngoc, Wuyts, Eva, Weldegebriel, Zerihun Berhane
• Format: Article
• Language: English
• Subjects: Alumni; International education; Most Significant Change; Multi-level impact; Multicultural; Theory of change
• ISSN: 0149-7189
• DOI: 10.1016/j.evalprogplan.2023.102228

To understand the impact of the international Master’s programmes offered at the Institute of Development Policy (University of Antwerp), a theory-based evaluation was undertaken. In the first phase, a Theory of Change (ToC) was elaborated, distinguishing between three levels of impact (individual, organisational and societal), four learning dimensions (knowledge, skills, attitudes and networks) and five implicit pathways (change agent, social network, widening access, academic diversity, international understanding). Given the multifaceted and vague nature of the ‘impact’ concept under study, we selected an international, gender-balanced, multi-sectoral team of alumni researchers who fostered inclusiveness of different perspectives, at the same time capitalising on their depth of understanding, having gone through the study experience themselves. A mixed-methods approach was adopted to validate the ToC, combining a Most Significant Change approach, categorisation and text analysis of 101 alumni impact stories. Our findings demonstrate the importance of the three levels of impact and four learning dimensions in capturing graduate impact. While the impact stories confirmed the dominant ‘change agent’ pathway, they also hinted at the importance of hybrid complementary configurations of pathways to fully grasp how impact materialises. •A theory based evaluation approach was used to unpack development impact of 3 international Master’s programmes.•A ToC with three levels , four learning dimensions , and five implicit impact pathways was elaborated and validated.•A multi-method approach using Most Significant Change, categorization and text analysis was used to validate the ToC.•An international, gender balanced, multi sectoral team of alumni facilitated multi-perspectivity.•The change agent impact pathway is dominant but other hybrid constellations of pathways can explain impact.