Exploring Qualitative Training Reactions: Individual and Contextual Influences on Trainee Commenting

• Published in: Journal of applied psychology 2015-05, Vol.100 (3), p.894-916
• Main Authors: Harman, Reanna Poncheri, Ellington, J. Kemp, Surface, Eric A, Thompson, Lori Foster
• Format: Article
• Language: English
• Subjects: Adult; Applied psychology; Data analysis; Employment - psychology; Evaluation; Feedback, Psychological; Human; Humans; Individual Differences; Individuality; Organizational behavior; Personnel Training; Qualitative analysis; Studies; Survey analysis; Surveys; Teaching - psychology; Training; Variance analysis
• ISSN: 0021-9010; 1939-1854
• DOI: 10.1037/a0038380

Training reactions are the most common criteria used for training evaluation, and reaction measures often include opportunities for trainees to provide qualitative responses. Despite being widely used, qualitative training reactions are poorly understood. Recent trends suggest commenting is ubiquitous (e.g., tweets, texting, Facebook posts) and points to a currently untapped resource for understanding training reactions. In order to enhance the interpretation and use of this rich data source, this study explored commenting behavior and investigated 3 broad questions: who comments, under what conditions, and how do trainees comment? We explore both individual difference and contextual influences on commenting and characteristics of comments in 3 studies. Using multilevel modeling, we identified significant class-level variance in commenting in each of the 3 samples of trainees. Because commenting has only been considered at the individual level, our findings provide an important contribution to the literature. The shared experience of being in the same class appears to influence commenting in addition to individual differences, such as interest in the topic (Studies 1 and 2), satisfaction (Studies 2 and 3), and entity beliefs (Study 3). Furthermore, we demonstrated that item wording may have an impact on commenting (Study 3) and should be considered as a potential lever for training professionals to influence commenting behavior from trainees. Training professionals, particularly those who regularly administer training evaluation surveys, should be aware of nonresponse to open-ended items and how that may impact the information they collect, use, and present within their organizations.