DiGen: Distractor Generator for Multiple Choice Questions in Code Comprehension

We propose an automated tool to assess code comprehension using Multiple-Choice-Questions. The core of our tool, DiGen, is a Named Entity Recognition model and an instructor-created database of well-commented code (whole or partial programs). For each code entry E in the database, DiGen creates a mu...

Full description

Saved in:
Bibliographic Details
Main Authors: Vimalaksha, Anusha, Prekash, Abhijit, Kumar, Viraj, Srinivasa, Gowri
Format: Conference Proceeding
Language:English
Subjects:
Assessments
Code Comprehension
Codes
Education
Generators
Machine Learning
Multiple Choice Questions
Named Entity Recognition
Natural Language Processing
Natural languages
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose an automated tool to assess code comprehension using Multiple-Choice-Questions. The core of our tool, DiGen, is a Named Entity Recognition model and an instructor-created database of well-commented code (whole or partial programs). For each code entry E in the database, DiGen creates a multiple-choice variant of an 'Explain in Plain English' (EiPE) question to assess learners' comprehension of code E. The correct choice is based on tags derived by the model from comments of E, and DiGen automatically generates distractors based on similar tags in the database, with pre-and post-processing routines to further improve distractor quality. To determine the efficacy of DiGen, we asked ten learners to compare the quality of the distractors generated by our tool against another tool that identifies distractors based on both comment similarity and code similarity. Our initial results suggest that our simpler approach leads to a slight improvement in distractor quality.
ISSN:2470-6698
DOI:10.1109/TALE52509.2021.9678662