Influence of overstated abstract conclusions on clinicians: a web-based randomised controlled trial

• Published in: BMJ open 2017-12, Vol.7 (12), p.e018355
• Main Authors: Shinohara, Kiyomi, Aoki, Takuya, So, Ryuhei, Tsujimoto, Yasushi, Suganuma, Aya M, Kise, Morito, Furukawa, Toshi A
• Format: Article
• Language: English
• Subjects: Access to information; Behavior modification; Bias; Biomedical Research - standards; Cancer; Clinical medicine; Clinical trials; Double-Blind Method; Evidence Based Practice; Evidence-based medicine; Female; Humans; Internet; Japan; Male; Physicians, Primary Care; Primary care; Psychiatry; Publications - standards; Researchers; Surveys and Questionnaires; Websites
• ISSN: 2044-6055; 2044-6055
• DOI: 10.1136/bmjopen-2017-018355

ObjectivesTo investigate whether overstatements in abstract conclusions influence primary care physicians’ evaluations when they read reports of randomised controlled trials (RCTs)DesignRCT setting: This study was a parallel-group randomised controlled survey, conducted online while masking the study hypothesis.ParticipantsVolunteers were recruited from members of the Japan Primary Care Association in January 2017. We sent email invitations to 7040 primary care physicians. Among the 787 individuals who accessed the website, 622 were eligible and automatically randomised into ‘without overstatement’ (n=307) and ‘with overstatement’ (n=315) groups.InterventionsWe selected five abstracts from published RCTs with at least one non-significant primary outcome and overstatement in the abstract conclusion. To construct a version without overstatement, we rewrote the conclusion sections. The methods and results sections were standardised to provide the necessary information of primary outcome information when it was missing in the original abstract. Participants were randomly assigned to read an abstract either with or without overstatements and asked to evaluate the benefit of the intervention.Outcome measuresThe primary outcome was the participants’ evaluation of the benefit of the intervention discussed in the abstract, on a scale from 0 to 10. A secondary outcome was the validity of the conclusion.ResultsThere was no significant difference between the groups with respect to their evaluation of the benefit of the intervention (mean difference: 0.07, 95% CI −0.28 to 0.42, p=0.69). Participants in the ‘without’ group considered the study conclusion to be more valid than those in the ‘with’ group (mean difference: 0.97, 95% CI 0.59 to 1.36, P