Development and validation of a scientific (formal) reasoning test for college students

We present a multiple‐choice test, the Montana State University Formal Reasoning Test (FORT), to assess college students' scientific reasoning ability. The test defines scientific reasoning to be equivalent to formal operational reasoning. It contains 20 questions divided evenly among five type...

Full description

Saved in:
Bibliographic Details
Published in:Journal of research in science teaching 2019-11, Vol.56 (9), p.1269-1284
Main Authors: Kalinowski, Steven T., Willoughby, Shannon
Format: Article
Language:English
Subjects:
Ability
Abstract Reasoning
Biology
College Science
College Students
Correlation
Discrimination
Evolution
formal operational reasoning
Hypothesis Testing
instrument
Introductory Courses
Multiple Choice Tests
Natural selection
postsecondary
Probability
Psychometrics
Quantitative psychology
Reliability
Science Curriculum
Science Instruction
Science Tests
Scientific Concepts
Scientific reasoning
Scores
Student Attitudes
Test Construction
Test Reliability
Test scores
Thinking Skills
Universities
Validity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a multiple‐choice test, the Montana State University Formal Reasoning Test (FORT), to assess college students' scientific reasoning ability. The test defines scientific reasoning to be equivalent to formal operational reasoning. It contains 20 questions divided evenly among five types of problems: control of variables, hypothesis testing, correlational reasoning, proportional reasoning, and probability. The test development process included the drafting and psychometric analysis of 23 instruments related to formal operational reasoning. These instruments were administered to almost 10,000 students enrolled in introductory science courses at American universities. Questions with high discrimination were identified and assembled into an instrument that was intended to measure the reasoning ability of students across the entire spectrum of abilities in college science courses. We present four types of validity evidence for the FORT. (a) The test has a one‐dimensional psychometric structure consistent with its design. (b) Test scores in an introductory biology course had an empirical reliability of 0.82. (c) Student interviews confirmed responses to the FORT were accurate indications of student thinking. (d) A regression analysis of student learning in an introductory biology course showed that scores on the FORT predicted how well students learned one of the most challenging concepts in biology, natural selection.
ISSN:0022-4308
1098-2736
DOI:10.1002/tea.21555