Loading…
Rigorous large-scale educational RCTs are often uninformative: Should we be concerned?
There are a growing number of large-scale educational Randomized Controlled Trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the EEF (UK) and...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Default Article |
| Published: |
2019
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/2134/36789 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1818168902169919488 |
|---|---|
| author | Hugues Lortie-Forgues Matthew Inglis |
| author_facet | Hugues Lortie-Forgues Matthew Inglis |
| author_sort | Hugues Lortie-Forgues (7157774) |
| collection | Figshare |
| description | There are a growing number of large-scale educational Randomized Controlled Trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the EEF (UK) and the NCEE (US) which evaluated interventions aimed at improving academic achievement in K-12 (141 RCTs; 1,222,024 students). The mean effect size was 0.06 standard deviations (SDs). These sat within relatively large confidence intervals (mean width 0.30 SDs) which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational RCTs often find small and uninformative effects. |
| format | Default Article |
| id | rr-article-9369251 |
| institution | Loughborough University |
| publishDate | 2019 |
| record_format | Figshare |
| spelling | rr-article-93692512019-03-11T00:00:00Z Rigorous large-scale educational RCTs are often uninformative: Should we be concerned? Hugues Lortie-Forgues (7157774) Matthew Inglis (1384290) Other education not elsewhere classified educational policy evaluation meta-analysis program evaluation There are a growing number of large-scale educational Randomized Controlled Trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the EEF (UK) and the NCEE (US) which evaluated interventions aimed at improving academic achievement in K-12 (141 RCTs; 1,222,024 students). The mean effect size was 0.06 standard deviations (SDs). These sat within relatively large confidence intervals (mean width 0.30 SDs) which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational RCTs often find small and uninformative effects. 2019-03-11T00:00:00Z Text Journal contribution 2134/36789 https://figshare.com/articles/journal_contribution/Rigorous_large-scale_educational_RCTs_are_often_uninformative_Should_we_be_concerned_/9369251 CC BY-NC-ND 4.0 |
| spellingShingle | Other education not elsewhere classified educational policy evaluation meta-analysis program evaluation Hugues Lortie-Forgues Matthew Inglis Rigorous large-scale educational RCTs are often uninformative: Should we be concerned? |
| title | Rigorous large-scale educational RCTs are often uninformative: Should we be concerned? |
| title_full | Rigorous large-scale educational RCTs are often uninformative: Should we be concerned? |
| title_fullStr | Rigorous large-scale educational RCTs are often uninformative: Should we be concerned? |
| title_full_unstemmed | Rigorous large-scale educational RCTs are often uninformative: Should we be concerned? |
| title_short | Rigorous large-scale educational RCTs are often uninformative: Should we be concerned? |
| title_sort | rigorous large-scale educational rcts are often uninformative: should we be concerned? |
| topic | Other education not elsewhere classified educational policy evaluation meta-analysis program evaluation |
| url | https://hdl.handle.net/2134/36789 |