Managing educational assessments with large groups of students can be tricky. At D2L, we evaluated a way to make that process a little easier in our Discussions tool.
-
Usability test
Semi-structured interview
-
UX Researcher
-
September - October 2022
Hereās an overview of my processā¦
š Context
D2Lās Brightspace is a learning management software supporting the delivery of educational courses. The Discussions tool helps users create and manage discussions within a course. Access to discussion topics can be restricted based on āgroupsā, āsectionsā, or āgroup categoriesā.
As part of a larger initiative to improve activity creation across Brightspace, this work focused on evaluating a design solution for the Discussions restriction options within the current design system.
What are āgroupsā and āsectionsā?
Simply put, āgroupsā and āsectionsā an organizational system for users (instructors, system admins) to:
organize and manage learners in Brightspace courses using āsectionsā
facilitate projects and activities (e.g, discussion posts) using āgroup categoriesā and āgroupsā
Option Functionality
When a user creates a discussion, they can limit which āsectionā, āgroup categoryā, or āgroupā can access and interact with a specific discussion. Basically, they can restrict who sees it or allow everyone to see it.
Use Case:
In Section 1, the instructor assigns discussion posts for homework. The class is divided into two group categories, the discussion topics (āDiscussion 1ā and āDiscussion 2ā). Learners are placed into groups to complete the work, (e.g., āGroup Aā, āGroup Bā) and can only access their assigned topic.
Designs of the group/section restriction options (prior to this study).
šThe Problem
The layout of information and availability of the discussion restriction options was inconsistent:
Despite having similar functionality, the options were located in separate areas of the Discussions tool. This was unintuitive - users had to search for the option they needed.
Even after finding them, users struggled to tell the options apart and use them successfully (lack of information, confusing copy, etc).
šÆ Research Objectives
Purpose: Evaluate the design solution to identify problems, opportunities for improvement, and learn about the userās behaviours and preferences regarding the restriction options.
These are some of the research questions I developed to guide my research:
Are participants able to use and understand restrictions related to groups and sections?
How do participants use groups & sections in Brightspace and the Discussions tool?
Is the ārate postsā functionality easily discoverable for participants?
Is the information displayed in the side panels understandable?
ā Method
Cross-Functional Collaboration
My role in this work was to execute and lead all aspects of the research, working in tandem with a product designer who oversaw the development and iteration of the proposed design solution. While I led the research efforts, the designer gave their expertise about the design solution and grounded my understanding of the impact this solution would have within the larger design system of the Discussions tool.
Approach
Choosing a method: A formative usability test was chosen because it was the most appropriate for the current stage of the design process (prototyping/testing phase). A semi-structured interview was included to gather data about participant data and their experience with groups and sections. The interview was intended to help me understand the recruited user segments deeper, and make sure I really understood their familiarity with groups and sections usage in Brightspace and Discussions. These experiences could provide some context for how users might approach interactions in the prototype, pain points, etc.
Communication: Our stakeholders were alerted for every major milestone of the project, and we made sure to involve them in user sessions via observation and sharing preliminary findings while the final report was in development.
Format
Sessions: Remote, moderated user sessions (via Zoom)
Semi-structured interview: Focus on learning about participantsā background, and familiarity and usage of groups and sections.
Usability test: Participants completed 5 tasks in a high-fidelity Adobe XD prototype (desktop).
šš»āāļø Recruitment
10 participants from 2 specific D2L user segments were recruited:
Admins: People who support the administration and usage of Brightspace in an organization.
Instructors: People who teach students in a course using Brightspace.
Why?
These segments typically have experience with discussion or course creation. Theyāre likely to encounter the redesigned restriction options in Brightspace because Admins need to organize students into course sections, and Instructors need to create/restrict discussions for students.
Data Analysis
1) Participant-by-participant analysis
Participants were analysed individually because they had different levels of familiarity using groups and sections. I wanted to find out if this impacted their interactions with the prototype.
(Spoiler: it did!)
2) Task-by-task analysis
I used tables in Miro to get a holistic view of the data across participants and the tasks completed in the prototype. Synthesis uncovered high-level insights and recommendations for design improvements.
Fun fact: Using tables and symbols helped me visualize the data and reduce cognitive load during analysis!
Key Findings
š Major usability issue: The difference between the restriction options was unclear for participants due to a lack of context.
Participants struggled to understand confusing terms like "threads" or "group category", and it was difficult to understand the option functionality. Participants weren't getting a lot of context from the copy on screen, and there weren't affordances to help alleviate their confusion.
This resulted in a lack of understanding about what each option did. Some participants were confused about how the restriction selection would impact the discussion they were creating, making it difficult for them to proceed with the task.
And the result of my work? A new Discussions experience! š¬
Implemented in 2023-2024, my research informed an updated discussion creation experience aiming to simplify complex tasks and make advanced Discussion features more accessible. This research was also part of a larger business initiative to bring consistency to activity creation features across Brightspace like the Discussions tool.
Here are a couple of recommendations I generated to address usability issues in the design:
My recommendations have been incorporated into the updated Discussions tool on Brightspace, now used by people who create and manage discussions!
Looking at the tested design compared to the live product, there is a significant different in the description copy of the discussion options. Greater detail and context about object functionality has been provided for users!