An Approach to Skill Mapping in Online Courses

Sean Lip, Dawn Zimmaro, Ross Strader, Norman Bier, Candace Thille

The Open Learning Initiative (OLI) is an open educational resources initiative that seeks to build learning environments based on principles derived from learning science research into how people learn. One of the main principles is that goal-directed practice and targeted feedback are critical to learning. Goal-directed practice involves working toward a specific level of performance and continually monitoring performance relative to clearly defined goals. When these goals are explicitly communicated to students, they guide and support students’ purposeful practice and help them monitor their progress. (Ambrose et al., 2010).

In order to create online learning environments that provide goal-directed practice, OLI courses start with the development of a skills map. The skills map is authored by domain experts who identify learning objectives and the skills that compose those objectives. Learning objectives specify what students will be able to do or know at the end of a section of instructional content, typically defined as a module. A skill identifies the discrete concept, or knowledge component, embedded within the learning objective. Each learning objective comprises one or more skills. Interactive activities and quizzes are created to assess students’ learning related to the various skills identified in the skills map.

The creation of skills maps serves a number of purposes, including assisting in the iterative course improvement process; measuring, validating and improving the model of student learning that underlies each course; and offering information necessary to support learning scientists in making use of OLI datasets for continued research. One of the best known uses of these skills maps, however, is to support learning analytics for instructors. In this context, the objective of skill mapping in OLI is to determine the probability that a given student has learned a given skill. Individual skills are treated as mathematically independent variables.

In the current version of the model used in OLI courses, student learning is modeled using a Bayesian hierarchical statistical model with the latent variables of interest, students’ learning states, becoming more accurate as more data is accrued about performance on a given skill (Lovett, 2012). This learning model is used to drive dashboard-style displays for both instructors and students. For instructors, the display gives a high-level overview of how students are performing on the learning objectives for each module in the course. Before going into class, instructors can see quickly the concepts students are grasping and the concepts with which they are struggling. This enables instructors to spend their time with students addressing areas where students are having trouble, as opposed to covering material that students have been able to master on their own (Strader et al., 2012). For students, the display shows level of mastery for each learning objective, and also allows students to view activities associated with each objective in order to provide an easy way to get more practice on a given objective.

We will give an overview of the skills map process, discuss the analytics reports and dashboard-style displays that skills mapping enables, review next steps for future improvements, and discuss implications for how skills mapping can drive student learning and iterative course improvement in MOOCs and other online courses.

References
Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K. (2010). How Learning Works: 7 Research-Based Principles for Smart Teaching. San Francisco: Jossey-Bass.

Lovett, M. (2012, July). Cognitively informed analytics to improve teaching and learning. Presentation at EDUCAUSE Sprint, July 25, 2012: http://www.educause.edu/ir/library/powerpoint/ESPNT122Lovett.ppt

Strader, R., Thille, C. (2012). The Open Learning Initiative: Enacting Instruction Online. In D. Oblinger, Game Changers: Education and Information Technologies (pp. 201-213). Washington, D.C.: EDUCAUSE.

Leave a Reply

Your email address will not be published. Required fields are marked *