Intuitive Test Theory

• Published in: Phi Delta Kappan 2005-03, Vol.86 (7), p.488-497
• Main Authors: Braun, Henry I., Mislevy, Robert
• Format: Article
• Language: English
• Subjects: Accountability; Achievement tests; Astronomy; Beliefs; Bruner, Jerome; Calculus; Cultural psychology; Earths Moon; Education policy; Educational Assessment; Educational evaluation; Educational leadership; Educational psychology; Educational tests; Educational tests and measurements; Evaluation Methods; Examinations; Expertise; Gardner, Howard; Grading (Education); Grading and marking (Students); High Stakes Tests; Kahneman, Daniel (1934-2024); Methods; Newtons law of cooling; Novices; Physics; Pickup trucks; Psychologists; Psychology; Reasoning; Resistance (Psychology); Social Behavior; Student Evaluation; Test scores; Test Theory; Testing; Testing Programs; Theory of impetus
• ISSN: 0031-7217; 1940-6487
• DOI: 10.1177/003172170508600705

Many of us have an intuitive understanding of physics that works surprisingly well to guide everyday action, but we would not attempt to send a rocket to the moon with it. Unfortunately, the authors argue, our policy makers are not as cautious when it comes to basing our school accountability system on intuitive test theory. Intuitive physics works well enough for playing catch with your dog or for building a birdhouse. But it doesn't work for constructing a bridge or shooting a rocket to the moon. One aspect of becoming an expert in physics is learning more sophisticated ways of thinking, but another is knowing when you need to use them, and yet another is recognizing when they fail. (Science is also about telling stories, but they are stories that submit to reality checks.) To Americans who go to school or hold jobs in the 21st century, taking tests is an experience nearly as familiar as pushing boxes or watching things fall. So we need to tell stories about tests -- their purposes, their construction, our performances on them -- and we need concepts to do so. While intuitive test theory is sufficient for classroom testing and for the quizzes in Seventeen magazine, it gets you into trouble when you want to evaluate performance on simulation-based activities, run a high- stakes testing program, or measure change in populations using an achievement survey like NAEP.