UX Research - Learning Management System Project
Design Brief
How might we transition from an old learning management system (LMS) to a system that is integrated into our workflow and meets the training and learning needs of our new hires as well as our existing employees thoroughly, encouraging learning and development throughout the organization? This capstone project was completed at a financial software company that recently migrated to a new learning management system to better facilitate learning of job responsibilities for new employees, ongoing responsibilities of current employees and growth learning for employees looking for their next move. We ensured that all stakeholders who use the platform, namely new employees, existing employees, instructors, managers, and the leadership team can easily access what they need to serve our employees (and therefore our customers) with effectiveness.
Provisional Methods
To learn how various employees interact with the LMS in the way that we have structured, branded, and organized it, I used three methods to conduct this research; semi-structured interviews, participant observation and usability testing. While it is an out-of-the-box product, there are customizations we can make. For instance, we can adjust the home page or make changes to the way we organize content for easier consumption to make it more usable for our employees. The tasks I asked of participants varied depending on their role (ie. I asked managers to assign training to their direct reports, and look at their progress, while I asked course administrators to edit aspects of their course and individual contributors to search and enroll in courses, and move through a course.) I found people in each role I was looking for across departments and geographic locations and did two rounds of ten UX research activities.
Findings
In the first round of interviews, usability tests and observations, I focused more on training methods and how we can ensure that we are training folks to use the tool in a way that addresses anything confusing. For instance, because it is a SharePoint product, the search functionality searches through all of SharePoint, meaning if someone is looking for a course on a tool, they might instead find a document that is used in a course. Another example is that because we have organized content into catalogs by department, a manager needs to select the correct catalog, (which is in a place that isn’t intuitive,) before they can assign the appropriate training. These are aspects of the tool that we need to train folks on. We also worked through some questions that our team didn’t internally discuss because it wasn’t on our radar before talking with other stakeholders in the company, like whether or not it is important to lock courses for folks who might not be ready for sensitive information about our system (or conversely, why we should have completely open courses.)
While we are still learning how much we have control over in the system, the interviews also helped us prioritize what questions we have for the platform in terms of how we can customize and maximize use of our tool internally. We plan on asking the following questions from our rep:
I was also able to prioritize some of the questions we need to ask ourselves internally surrounding how we build our training system:
While we are still learning how much we have control over in the system, the interviews also helped us prioritize what questions we have for the platform in terms of how we can customize and maximize use of our tool internally. We plan on asking the following questions from our rep:
- On the manager dashboard, it is unclear that there are buttons to click next to the names of direct reports. It only appears when you hover so you don't know that you can select them, or than you can select more than on.
- Can we make the pictures at the top of each page smaller? It is taking up too much precious real estate and they would like more room to actually scroll.
- Can we hover over titles to get an explanation (Ie. CEU’s, competencies, etc.)
- Can we change the “Enroll” Button to the top of the screen instead of the bottom?
- For Blended Courses, most people thought the reason that the Enroll button was greyed out was because they didn’t have permission to enroll or they hadn’t completed a requisite. Can we include instructions that they need to select a session in order to enroll?
- Can a manager see the courses that their direct reports are admins for?
- What does Following or Not Following mean?
- At the bottom of the page it allows you to save for later, but unclear how I actually get to the saved content. If I hover it says “See recently saved items” but I can’t click on it.
- Is there any way to get courses to always open in new tabs so that it is easy to always get back to the dashboard?
- Can we change the fonts at all? They were too small and hard to read. Furthermore, the tab colors are a little too similar for folks so it was a little hard to tell the difference between the tabs to know what was selected.
- Can I have a minimum # of people enrolled in a course? (I know I can have a max)
- Can we assign more than one course at a time? Also, can someone enroll in more than one course at a time.
- Is there a way to have LMS 365 generate a Teams link for a blended course?
- Why don’t events show up if we have some scheduled? Is there a way to connect the blended courses to the events?
- Can we not add due dates for classroom/ blended courses?
- Anywhere that it says ‘LMS365,’ can we change it to ‘Compass’?
I was also able to prioritize some of the questions we need to ask ourselves internally surrounding how we build our training system:
- Should we have instructors record attendance? If so, how is that information used?
- How should we implement deadlines to help folks prioritize what they need to do?
- It is unclear what is mandatory and what might not be. Let’s think about how to incorporate ways to show aspects of courses that are optional.
- What notifications do we want managers and employees to be receiving?
- We should specify that if someone doesn’t have access to one of the online learning platforms, they won’t be able to access those courses.
- Can we just have a general link to the job catalog and online learning platforms? We should also link to the Role-Based Learning Paths as they exist now. They need to be updated but it might be helpful to just link to them for now.
- Should we have more paths based on how senior or how experienced someone is?
- Quizzes are too long.
- Think about building out CEU’s – Can we promote those courses so that people can keep track of them more easily (like create separate courses to send folks to the appropriate online learning platform courses, maybe?
Consent Process
I am attaching my consent form, which outlines that participants will not be paid (extra) for their participation in my study, what I am looking to gain from their interview, usability test, or observation, and that they are free to decline to answer or participate at any time. I sent all twenty participants the consent form along with the invitation at least a day or two in advanced, along with the following message:
Hello,
You may or may not know that the Talent Enablement team is switching our training content from Litmos to our new learning management system, Compass. In an effort to ensure that it is meeting the needs of our users, we are conducting some user testing over the course of the next couple of weeks. We are including different kinds of users: new employees, existing employees, managers, and course admin from different offices and departments. I have chosen you as a subject for this testing if you agree.
I may assign you a course or training plan to help give you some context (so you may get an email saying it has been assigned to you,) but there is nothing you need to do ahead of our meeting.
Additionally, I am using this as a capstone project for a credential in UX research. Attached is a consent form explaining what I will be doing. If you consent to a 20 minute interview, usability test, or observation (specified in your invite,) please respond that you consent. I am looking forward to speaking with you. Let me know if you have any questions.
Thanks.
At the beginning of each call, I asked if participants were able to review the consent form. Most of them had, and I reiterated that they could stop at any time if they felt uncomfortable or wanted to stop being a participant for any reason. I also asked if they consented to my recording of the conversation. I had no participants who had any concerns around the consent process, other than unfamiliarity with a consent form for participation in this kind of activity. (Many people thought that by the nature of what I sent them, that I might be asking much more personal or sensitive information.)
All participants verbally consented and also sent an email saying in writing that they consented to everything in the consent form that I sent.
Hello,
You may or may not know that the Talent Enablement team is switching our training content from Litmos to our new learning management system, Compass. In an effort to ensure that it is meeting the needs of our users, we are conducting some user testing over the course of the next couple of weeks. We are including different kinds of users: new employees, existing employees, managers, and course admin from different offices and departments. I have chosen you as a subject for this testing if you agree.
I may assign you a course or training plan to help give you some context (so you may get an email saying it has been assigned to you,) but there is nothing you need to do ahead of our meeting.
Additionally, I am using this as a capstone project for a credential in UX research. Attached is a consent form explaining what I will be doing. If you consent to a 20 minute interview, usability test, or observation (specified in your invite,) please respond that you consent. I am looking forward to speaking with you. Let me know if you have any questions.
Thanks.
At the beginning of each call, I asked if participants were able to review the consent form. Most of them had, and I reiterated that they could stop at any time if they felt uncomfortable or wanted to stop being a participant for any reason. I also asked if they consented to my recording of the conversation. I had no participants who had any concerns around the consent process, other than unfamiliarity with a consent form for participation in this kind of activity. (Many people thought that by the nature of what I sent them, that I might be asking much more personal or sensitive information.)
All participants verbally consented and also sent an email saying in writing that they consented to everything in the consent form that I sent.
Documentation
This is the slide deck created to help employees understand how to use our new system. Some information was removed for company privacy.