Elahe Rostami
Hybrid Discovery Usability Research
Client: Canada Post Marketing and Business product teams
Test artifacts: New and legacy self-serve marketing tool
March 2024
Background:
The clients business and marketing team planed to decommission a legacy high value self-serve digital tool and transition users to an updated version of the offering if usage reaches critical mass. However the teams are measuring a lower then expected adoption rate on analytics as well a high bounce rate from the updated tool. The Voice Of Customer team collected data highlighting usability and performance issues, prompting further research to facilitate the transition and understand user demographics.
Research Objectives
A multi-stream research approach was recommended to achieve all these objectives.
1. Support the Marketing and business team in investigating why only half of users who begin the flow opt for the updated marketing tool over the legacy.
2. Understand why 3/4 of new tool users drop off before completing the task.
3. Determine customer and persona segments using these specific tools today.
A multi-stream research approach ran concurrently to achieve these objectives.
Research Stream 0
Desk Research
Summarize insights from previous Qualtrics surveys on classic and new tool pages and shed light on:
• User preferences and the reasons behind their choices between the new and classic flows.
• Identifying gaps in knowledge and areas requiring further research.
Research Stream 1
Qualtrics Surveys
Objectives:
• To understand who the range of users are, that have been using both the classic and newtools, leading towards first level user segmentation.
• To get a first level understanding of user's goals for being on the platform as well as general satisfaction/ high severity issues that they may have been facing.
• To gage attitudes towards offerings and factors that may contribute to preferences for features between classic vs. updated tools .
Method:
Target Users/Customers were recruited through Qualtrics intercept surveys on the signed-in state pages, as well as customer lists provided by the product teams .
Deliverables:
Summary insights debrief meetings summarizing roles that are using the tools, their goals, satisfaction with, and user preferences between new and classic flow.
Research Stream 2
Hybrid Discovery Usability Interviews
The study tested usability and desirability of a new tool, part of Canada Post Website to support the Marketing and Business team with the following objectives:
1.To obtain feedback on updated tool, understand severity of usability issues and to build on prior gathered insights and investigate team’s hypothesis around high drop off rate.
2.To gain a deeper understanding of user goals, mental models, workflow, and value as it relates to different user segments within their order entry journeys.
3.To gage attitudes towards offerings and factors that may contribute to preferences for features between the Classic vs. Updated tool.
Method:
Hybrid moderated semi-structured discovery and usability Interviews were conducted remotely on User Testing platform.
In addition, an unmoderated study with similar structure was set up on User Testing platform to accommodate for participants with time restrictions.
Interview structure included two parts:
1- The discovery part: aiming to understand participant’s background, context of use, goals, mental model and prior experiences with specific tools.
2- Usability assessment part: User feedback was collected as participants visually navigated through platforms, simulating a real-world scenario and providing their thoughts and feedback.
After each 1 hour interview the team gathered for 15-30 mins debrief workshops to collect their observations and thoughts on a Miro board to be categorized and taken account in final synthesizes.
04
Segmentation and Recruitment
After conducting a segmentation alignment workshop with the teams on Miro target proto personas were identified.
A representative segment of participants were interviewed from segments (Entrepreneurs, Campaign Managers, Print Partners, and Creative Agencies ). Participants were recruited through the User Testing Panel, VOC Qualtrics surveys, and direct recruitment through the product team’s contacts with partners.
Synthesis Analysis of Data
Findings Report
Synthesized findings from all research streams were presented to stakeholders across the organization. The 90-page final findings report included sections on:
01
Target users segmentation contextual inquiry
Psychographic and firmographic data of the four identified proto segments were captured through the discovery part of the interviews. Insights on workflow, goals, prior tool experiences, trends, themes, and expectations were communicated.
03
Detailed Usability Observation
Actionable insights on usability issues including navigation and performance pain points across a spectrum of severity as well as positive and noteworthy feedback were presented
5 issues of low severity
5 issues of medium severity
12 issues of high severity
5 Positive feedback
20 noteworthy insights
02
Pain points and Opportunities found around each objectives
To support the teams in understanding the contributing factors leading to measured behaviours(low adoption of the new tool and high drop-off rates), pain points and opportunities were reported separately and in relation to each segment. Screen captures and videos from user interviews were used in the report to create impact and support empathy building around user's needs.
04
Post Impact effort prioritization workshop
Worked with immediate product teams and managers through a post presentation Impact–Effort Matrix workshop to plot relative user value against implementation complexity for tackling findings.