Lessons and recommendations from three decades as an NSF REU site: A call for systems‐based assessment

• Published in: Ecology and evolution 2020-03, Vol.10 (6), p.2710-2738
• Main Authors: McDevitt, Andrew L., Patel, Manisha V., Ellison, Aaron M.
• Format: Article
• Language: English
• Subjects: Academic Practice in Ecology and Evolution; Cognition & reasoning; Collaboration; College students; Colleges & universities; cultural–historical activity theory; Data collection; Ecology; Forestry research; Harvard Forest; Laboratories; Learning; Program evaluation; research experiences for undergraduates; Researchers; STEM; STEM education; Students; Theory; undergraduate research experience
• ISSN: 2045-7758; 2045-7758
• DOI: 10.1002/ece3.6136

For more than 30 years, the US National Science Foundation's Research Experiences for Undergraduates (REU) program has supported thousands of undergraduate researchers annually and provides many students with their first research experiences in field ecology or evolution. REUs embed students in scientific communities where they apprentice with experienced researchers, build networks with their peers, and help students understand research cultures and how to work within them. REUs are thought to provide formative experiences for developing researchers that differ from experiences in a college classrooms, laboratories, or field trips. REU assessments have improved through time but they are largely ungrounded in educational theory. Thus, evaluation of long‐term impacts of REUs remains limited and best practices for using REUs to enhance student learning are repeatedly re‐invented. We describe how one sociocultural learning framework, cultural–historical activity theory (CHAT), could be used to guide data collection to characterize the effects of REU programs on participant's learning in an educationally meaningful context. CHAT embodies a systems approach to assessment that accounts for social and cultural factors that influence learning. We illustrate how CHAT has guided assessment of the Harvard Forest Summer Research Program in Ecology (HF‐SRPE), one of the longest‐running REU sites in the United States. Characterizing HF‐SRPE using CHAT helped formalize thoughts and language for the program evaluation, reflect on potential barriers to success, identify assessment priorities, and revealed important oversights in data collection. This article proposes using cultural–historical activity theory (CHAT) as a systems‐based theoretical framework for evaluating and improving experiential undergraduate research programs. After describing the CHAT framework, we illustrate its application to the collection and interpretation of data collected by the Harvard Forest Summer Research Program on recruitment and student hiring practices; understanding variation in learning gains; and assessing its long‐term impacts.