Selecting and assessing challenge problems

Organisations conducting research programs often focus the work of their scientists and technologists on challenge problems (CPs). These challenges are designed to ensure that progress is measurable and relevant to the goals of the program sponsor. Generating and selecting pertinent CPs is difficult...

Full description

Saved in:
Bibliographic Details
Published in:Theoretical issues in ergonomics science 2019-01, Vol.20 (1), p.27-38
Main Authors: Freeman, Jared, Leins, Drew, Bell, Conrad
Format: Article
Language:English
Subjects:
Challenge problem
Chemists
Interdisciplinary research
Interdisciplinary studies
Organic chemistry
problem definition
research policy
Research programs
Scientists
Technologists
voting
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Organisations conducting research programs often focus the work of their scientists and technologists on challenge problems (CPs). These challenges are designed to ensure that progress is measurable and relevant to the goals of the program sponsor. Generating and selecting pertinent CPs is difficult, as is assessing their value. We describe a method of generating and selecting CPs and its application in a highly collaborative, multi-organisation research program. Thirty-eight biologists, chemists, mathematicians and computer scientists across academic, commercial and government organisations generated and ranked their top choices from among 12 richly described candidate challenge problems. A ranked-choice voting formula was applied. Five CPs were highly scored; the remaining seven were distributed across a lower range of scores. The program sponsor subsequently directed researchers to address six CPs, including the elected five. Analysis of the rationales that participants offered for their CP rankings revealed four domain-independent dimensions of value: capability, speed, impact and synergy. These dimensions of value can help managers of interdisciplinary research programs systematically select a portfolio of CPs that will efficiently apply utilise resources towards program goals and facilitate measurement of scientific progress.
ISSN:1463-922X
1464-536X
DOI:10.1080/1463922X.2018.1485987