Aligning open educational resources to new taxonomies: How AI technologies can help and in which scenarios

• Published in: Computers and education 2024-07, Vol.216, p.105027, Article 105027
• Main Authors: Li, Zhi, Pardos, Zachary A., Ren, Cheng
• Format: Article
• Language: English
• Subjects: Alignment; Common core; Multimodal; Open educational resources; Taxonomy
• ISSN: 0360-1315; 1873-782X
• DOI: 10.1016/j.compedu.2024.105027

Aligning open educational resources (OER) to skill taxonomies is a common task in the education field and helps teachers better locate material that aligns with the standards of their curriculum. When taxonomies change, as they periodically do, re-tagging the increasing mass of open educational resources is needed. The process of manual tagging is, however, exceedingly labor intensive. We propose and evaluate a novel combination of machine learning methods to help automate tagging open educational resources with skills from an existing taxonomy as well as skills from any newly introduced taxonomy. We collected text, image figures, and videos from tens of thousands of educational resources from two major digital learning platforms to answer the research questions of: how effective are machine learning models in automatically updating OER classification to reflect a new taxonomy (RQ1), and which models may be of practical use in different scenarios (RQ2)? Using several taxonomies, including the US Common Core, we find that while full automation is not practically viable, our most generalizable model can reach non-expert human labeling performance requiring only 100 labeled examples and near expert level with 5000. We believe these novel findings may have immediate utility for practitioners and policymakers and better ready the growing landscape of open educational resources for the advent of new taxonomies ahead. We publicly release our pre-trained US Common Core and new taxonomy tagging models, providing guidance on their viability in various real-world scenarios. •Establishes a new state of the art in aligning problems, videos, and skills to taxonomies.•Introduces a novel task of aligning OERs to a new taxonomy that is not seen in training.•Finds zero-shot learning achieves practical accuracy aligning OERs to a new taxonomy.•Finds contemporary multi-modal techniques do not substantially improve alignment.•Releases pre-trained models and gives guidance on their utility in practical scenarios.