Development and Validation of an Assessment Instrument for Course Experience in a General Education Integrated Science Course

• Published in: Journal of geoscience education 2017-11, Vol.65 (4), p.435-454
• Main Authors: Liu, Juhong Christie, John, Kristen St, Courtier, Anna M. Bishop
• Format: Article
• Language: English
• Subjects: assessment instrument; CAI; Case studies; Classroom Desegregation; Classrooms; College Science; Computer assisted instruction; Construction standards; Core curriculum; Course Content; course experience; Distance learning; Earth Science; Education; Educational Environment; Educational Research; Educational technology; Electronic Learning; Experimental design; Factor Analysis; General Education; Higher education; Information literacy; Learning Processes; Learning Strategies; Learning Theories; Mathematics Education; Measures (Individuals); Nonmajors; On-line systems; Online Surveys; Polls & surveys; Population (statistical); Problem solving; Quality; Questionnaires; Reliability analysis; Reliability aspects; Resampling; Researchers; Science Curriculum; Science education; Skills; Stability analysis; Statistical Analysis; STEM Education; Student Attitudes; Student Experience; Student Surveys; Students; Studies; Teachers; Teaching; Teaching Methods; Test Construction; Test Reliability; Test Validity; Tourism; Undergraduate Students; Undergraduate Study; Validity; Working conditions; Workload; Workloads
• ISSN: 1089-9995; 2158-1428
• DOI: 10.5408/16-204.1

Identifying instruments and surveys to address geoscience education research (GER) questions is among the high-ranked needs in a 2016 survey of the GER community (St. John et al., 2016). The purpose of this study was to develop and validate a student-centered assessment instrument to measure course experience in a general education integrated science course. A one-shot case study of pre-experimental design (Creswell, 2009, 2014) was used to understand student experiences in a large-enrollment course with digital content integration, including out-of-class video, online presentations, and warm-up questions, as well as in-class video paired with discussion and small group activities. In two sections taught by the same geoscientist, 209 students accessed course content in an online learning management system prior to classroom instruction. We adapted the 36-item Course Experience Questionnaire (CEQ; Wilson et al., 1997) to a Web-based survey. We conducted statistical analysis on the CEQ responses, including item factor analysis, examination of communalities and measures of association, confirmatory factor analysis, and reliability and stability testing through bootstrap resampling. The statistical analyses indicate that the results in this study are comparable to those from other cultural contexts and subject areas. Given the moderate fit of the model and reasonably stable results, we propose that the core indicators of the CEQ constructs, including Appropriate Workload, Clear Goals and Standards, Generic Skills, Good Teaching, and Emphasis on Independence are sufficient to assess students' course experiences in general education science settings. These results provided moderately to strongly valid information that can help instructors in designing their technology-integrated classes, although further study with a larger sample population would be beneficial. From this study we conclude that students' perceptions of their course experiences are closely related to their development of problem-solving and analytical skills, clear course expectations to direct their own study plans, their preference for a motivating instructor, the opportunity to have a variety of learning choices, and their workload. We propose the use of a 25-item version of the CEQ that is appropriate for formative assessment in discipline-specific STEM introductory classes, including geoscience classes.