Members of the lab are involved in a number of ongoing projects. Some of the more prominent ones are described below.
Learning Analytics and Theory as Guides to Improve Undergraduate STEM Education
Undergraduate STEM courses are increasingly using learning management systems to provide extra supports for students and instructors. However, little research has been conducted on how best to structure these systems to provide students with metacognitive and motivational supports that might facilitate learning. The project includes three studies that 1) identify how students use the elements of the learning management system content and how that use relates to student outcomes; 2) test embedded learning strategy training experiences for students and motivational interventions to examine the affordance for student motivation, behavior, achievement, and completion rates; and, 3) test whether a behavior-based early warning system will identify problematic learning behaviors earlier and more accurately so that interventions might be targeted to the students particular problems.
Research is conducted in four courses in biology, algebra, calculus and two engineering domains in the first years of students’ undergraduate coursework. Researchers and course instructors collaboratively design the course materials hosted on the Blackboard Learn learning management system (LMS), the most common LMS currently in use.To capture student use of learning resources, we use Splunk – a powerful software platform for managing, observing, and modeling both historical and constantly accruing logs of data. Splunk tools enable us to conduct data mining analyses, and to build systems to reach out to students in close to real time.
Findings to date include:
- a number of papers that investigate STEM learners’ motivations and their relationship to behaviors and achievement.
- a two-hour commitment to completing web-based Science of Learning to Learn modules changed biology students’ learning behaviors, and improved their performance on course exams. Students who completed training made greater use of resources designed for planning and monitoring learning, and conducted more retrieval practice with online quizzes in the week prior to exams. Students in the training group typically outperformed students completing other learning activities by 3-5% on subsequent exams. Struggling learners, first generation college students, and students from under-reprepresented groups in universities and STEM programs obtained substantially stronger gains (as many as 7-9% on the final exam; ds = .50-.81)
- Using students’ LMS behavior data from the first four weeks of their biology course, a prediction model can correctly identify more than 80% of students who will ultimately earn a C or worse in the course. A prediction model for calculus achieves similar accuracy using only 3 weeks of data. These prediction models allow the team to send an early alert to students who appear to be struggling. Students receive this message from their instructor a week before the exam, and are given advice and other resources.
- Students who receive messages based on prediction models and who make use of the resources are doing better in their courses:
- Messaged students tend to beat their projections – 1 in 3 predicted to earn a C or Worse earn As and Bs in the course.
- Within a week of receiving the message from their instructor, alerted students outperform others projected to perform poorly – Differences on Exam 1 scores (8 days after a message) were 5%.
Educational Datamining under the supervision of Learning Theory: A Learning Sciences Collaboration investigating STEM Learning
In this collaborative project with the Lab of Dr. Andreas Stefik (Computer Science, UNLV), a pair of graduate research work collaboratively and with their advisors to use programming and learning analytic methods to investigate how individuals learn in math, science, and programming tasks. Using the rich datasets of machine data produced when users interact with software, researchers will examine the learning processes that can be inferred from learners’ behaviors and their association with learner characteristics and performance. Parallel projects will be conducted in Blackboard Learn-supported courses, Cognitive Tutor lessons in Algebra and Geometry, a programming task in Quorum, and courses using Docuwiki software.
This project is supported by an award from the UNLV Graduate College. It is scheduled to run from August 2014 to July 2017.
Building Capacity for Research on Learning via Analytics and “Big Data”
In this project, Lab members will work with the Office of Instructional Technology and the Teachable Agents Group at Vanderbilt University to adapt an existing differential sequence mining tool so that it can be used to test hypotheses about student learning in multiple learning environments. Tools will be designed to identify learning behaviors that are differentially common amongst learners using Cognitive Tutor,an intelligent tutoring system for mathematics, and in Blackboard Learn, a learning management system utilized at UNLV other colleges and universities. These tools will provide a basis for addressing two research questions:
- How do learner factors like prior knowledge and motivation influence learning behavior and performance when learning with ITSs and LMSs?
- Can instructional design approaches promote desired learning behaviors and outcomes?
This project was supported by an award from the UNLV Faculty Opportunity program. It was funded from August 2014 to March 2016. The sequencing tool created as part of the project can be found here.
Personalizing Algebra Learning
Lab members collaborate with Candace Walkington, a Math Education expert, to examine the mechanisms by which personalization of math instruction improves student learning. Candace has been a pioneer in the personalization of math problem solving exercise to students’ outside interests in sports, games, music, and food. Algebra students who learn using personalized materials tend to outperform those who receive more typical materials on a number of key learning outcomes. New studies involving Lab members are underway to investigate the roles that student interest and prior knowledge play in the development of algebraic reasoning. Additional studies examining the utility of problem-posing activities, which help students learn algebra by authoring their own algebra problems that incorporate their personal out-of-school interests. Much more about these studies can be found at Candace’s website.
Partial support for these projects comes from a subaward from the National Science Foundation via LearnLab, the Pittsburgh Science of Learning Center. Additional funding information is available at Dr. Walkington’s website.