Setting Up Your LMS to Get the Most From Your Data

Improved teaching practices are closely linked to improved learner outcomes. But whether your courses are in person or online, documenting and analyzing pedagogical practices – especially in real-time – is prohibitively tricky without significant assistance from technology. Luckily your Learning Management System (LMS) does a phenomenal job of capturing all kinds of data – so much data, in fact, that it can sometimes feel overwhelming. So how can you harness all that data to improve your pedagogy and what does that look like in practice? And how can you set up your LMS to facilitate it?

Paula de Waal is a world-class Learning Analytics expert who wrestles with these questions every day. Dr. Paula de Waal is the Head of Teaching Innovation at Universita Ca’Foscari Venezia, Italy. A passionate educator, she has a breadth and depth of knowledge in e-learning, educational technology, Learning Analytics, and instructional design. She is also actively involved in the Italian Moodle Users Association (AIUM) and uses the Moodle LMS and the IntelliBoard reporting and analytics plug-in to conduct her research. 

We’ve been proud to call Paula de Waal a client for the past four years and were honored to have her give a presentation on Design for Analytics at IntelliCon 2021. 

As part of her IntelliCon presentation, she shared some great foundational insights on setting up your Moodle LMS and IntelliBoard to get the right data to put educator-centered Learning Analytics to work in your courses. 

What is Learning Analytics?

Learning Analytics collects and analyzes data produced by learners during their interactions with online and offline learning environments and combines descriptive, diagnostic, and prescriptive analytics with the very human skills of instructional design, learner-centered advocacy, and communication. 

Dr. Paula de Waal is particularly interested in how Learning Analytics can be applied to pedagogy in real-time:

“I am still interested in solving critical issues in the use of resources and participation in learning activities, but during the ongoing course… what all [educators] look for all the time are methodologies and tools that allow them to reflect about the course that is in action. I was hoping that Learning Analytics would help in that direction. And of course, to save time with the documentation of processes, which is very, very, very time consuming if you want to analyze heavy, big data sets.”

Let’s say you, like Dr. Paula de Waal, use the Moodle LMS with the IntelliBoard reporting and analytics plug-in. You would pick which data points you want to be extracted from your courses and in what format and how frequently you’d want reports delivered via IntelliBoard so you wouldn’t have to spend a lot of time sifting through data and compiling spreadsheets. You would then analyze those reports to see what your students are doing as they’re doing it and make adjustments to your pedagogy in real-time based on what you’re seeing. How are your students interacting with their assignments? Are they spending the expected amount of time on key course materials or is it significantly more or less than you thought they would? Are there any notable course-wide or individual learner longitudinal trends or correlations? 

The variables you choose to track will vary based on the particular needs of your course and your learners, but however you choose to deploy Learning Analytics, they can be a powerful tool to gain insight into your pedagogy.  

Customize Tagging, Collecting, and Reporting Data

As educators, we have to think very carefully about how to collect and tag data for each type of student in each unique course to evaluate students’ interactions with their learning environment and our pedagogy in a meaningful way.

An educator leading a math course with an attendance requirement, a fairly homogeneous crop of learners, and multiple-choice assessments is going to want to track and report data very differently than an instructor teaching a language course where attendance is optional, learners are from a variety of programs, and coursework is discussion and essay-based.

Dr. Paula de Waal found that using a data reporting and visualization system like IntelliBoard can help communicate some of these contextual nuances to educators, learners, and institutions alike. 

“It’s very easy for people who are not trained in understanding this data to use it as it is without connecting that to [context] at hand. So when we started to study using IntelliBoard some of these problems disappeared, but we found other problems, which were, how do we fill Intelliboard with good data? […] what we tried to do at our university and through the courses that we helped to organize with the Moodle community is to teach the teachers how to use the variables of the platforms to set their own courses.”

Just as a one size fits all process doesn’t work for assigning useful metrics to courses, a one size fits all approach doesn’t work for putting together analytics reports that convey contextualized data meaningfully. You have to set up your LMS to populate IntelliBoard with the right data and schedule your Learning Analytics reports to be sent to the right people at the right intervals for your course.

How Do I Set Up My LMS to Get the Right Data?

Dr. Paula de Waal`s highlights these basics for setting up your LMS for success:

1. Pay attention to time tracking. Make sure you correctly configure delivery and task completion dates, course duration, and variables that allow for variations in delivery and course completion.

Dr. Paula de Waal’s top time tracking mistakes

2. Track competences. Make sure you include a framework that references each area of competence that students must meet. Knowing a student is struggling in your course by looking at their overall grade is good, but being able to break down each assessment into individual areas of competence and tracking those trends is a much better way to offer targeted help.

Dr. Paula de Waal’s top competence tracking mistakes

3. Don’t forget task tracking. Activate it, configure completion in individual tasks or resources so completion doesn’t register at first click, and make sure you leave any unnecessary tasks out of your tracking.

Dr. Paula de Waal’s top task and course completion tracking mistakes

4. Pay attention to roles and labels. Who is assigned what role in your course? Do people have multiple roles? Have you changed the role or just how it’s labeled? Think about it all ahead of time so you can make sure you’re getting useful contextualized results.

Getting these basics taken care of before your course begins will lay a strong foundation for getting actionable Learning Analytics that are tailor-made to measure what matters to you and your students and help you continuously improve your pedagogy and your students’ learning environment.

A Word of Caution: Avoid Context Agnostic, Top-Down Interventions

Context-awareness extends beyond how an individual educator tailors Learning Analytics for their particular course. It also matters how, when, and to whom data is communicated. If information is shared in such a way that either educators or learners feel it’s being used to penalize rather than help them, then that’s bound to interfere with the quality of learning. 

Dr. Paula de Waal emphasizes that data gathering and labeling shouldn’t increase the risk of top-down bureaucratic interventions that strip the data of its context. If educators believe that collecting specific data points will trigger counterproductive intervention on the part of their institutions, Learning Analytics lose a lot of their potential for a positive impact on continuous improvement of pedagogy and the learning environment. Data needs to be illuminative, not punitive and educators need to trust that their institutions view it as such. 

Course Design and Educator Empowerment to Harness the Power of Learning Analytics

If Learning Analytics are harnessed appropriately and a strategy for deploying them is baked into your course design and LMS configuration from the beginning, you can look at what matters to you, meaningfully measure how your students are doing in real-time, and continuously improve your pedagogy and the learning process to make each student’s learning experience the best it can be. 

Thank you, Dr. Paula de Waal, for your excellent presentation. We are excited to see where your research takes you as the field of Learning Analytics continues to evolve. We’ll be there with you every step of the way. 

Are you interested in harnessing Learning Analytics to improve your pedagogy? Take advantage of our webinars, or schedule a chat with us. We’d love to help you unlock the power of your data.

Watch the full video of Paula de Waal’s presentation below.