NDA Project
Electric Utility App —
Usability
Test for launched
Utility App
To validate the usability of the selected functions within the launched utility App, identify improvement areas through understanding of end customers’ pain points and identifying new opportunities; in order to enhance the user experience of each function within utility App.
The Whole Process.
1. Recruitment
2. Usability Interview
3. Qualitative Analysis
4. Ideation
Recruitment
-
Screening survey
-
Recruit usability study participants based on their responses to cover diversity
Usability Interview
-
User test and observation of user interaction and behaviors
-
Uncover pain points and opportunities
-
5 Whys method to dive deep into user minds and uncover root causes
-
Benchmark other apps and customer experience offered by other utility companies
Qualitative Analysis
-
Highlighting quotes
-
Affinity mapping and clustering
-
Mapping of pain points and opportunities to the existing user flows
Ideation
-
Ideate possible solutions
-
Evaluate and select best solutions as recommendations
Methodology.
Qualitative > Quantitative
For UI UX design, we tend to apply qualitative research method to uncover “why” and “how”, as well as diving deep into users’ psyche, instead of “what” as quantitative research provides us with. Hence, due to the nature of this project, our objectives are better achieved through qualitative research methods.
-
Inform design decisions
-
Identify usability issues
-
A few participants
-
Flexible study conditions based on team’s needs
Usability Study Approach.
Recruitment
45 responded to the screening surveys that were sent via Google Forms and 15 were recruited to participate in the usability interview based on their answers to the survey that is co-designed by my company, utility App's Public Affairs and Customer Services Division.
Why Are 5 Participants Per Segment Enough?
According to UX researchers, 85% of usability issues can be identified by doing a qualitative test with 5 testers. Not only does the rate of which issues can be identified plateau after the 5th tester, exceeding the threshold of users will make analysing data difficult since they are too dense.
Interviews.
1 ON 1 INTERVIEW.
15 in-depth face-to-face interviews (45 mins to 1 hr in length) regarding App’s core features within 2 weeks.
USABILITY TASKS.
3 tasks were designed to cover the core features based on the end-to-end cycle of an end user (i.e., Registration, managing account, and moving out of their homes).
Task A
AOL Registration &
SSO Binding
Task B
Payment Record &
e-Payment
Task C
Application
Progress Tracking
Qualitative Analysis.
Raw Data.
Pain Points & Opportunity Mapping.
Preview.
Usability Rate
3.5 / 5
AOL Registration &
SSO Binding
4.7 / 5
Payment Record &
e-Payment
2.3 / 5
Application
Progress Tracking
How Is Usability Rate Determined?
The usability rate is the average score given by 15 participants who were asked to rate the tasks’ ease of use on a scale of 1 to 5 (1 means lowest usability, and 5 means highest usability; usability refers to if users can perform the tasks effectively and efficiently while enjoying the experience) after completing each task.
Analyse by 5 Dimensions.
Insights & Recommendations.
Let's focus on Task C!
2.3 / 5
Application Progress Tracking
Submit an online move in application. Then check the status of your application that you have just submitted.
Submit an online move in application. Then check the status of your application that you have just submitted.
The usability rate is the average score given by 15 participants who were asked to rate the tasks’ ease of use on a scale of 1 to 5 (1 means lowest usability, and 5 means highest usability; usability refers to if users can perform the tasks effectively and efficiently while enjoying the experience) after completing each task.
53% Users (8/15) Tracked Application Progress via the Utility App
The rest of users* used other ways to access Application Progress Tracking.
*47% (7/15) of users used emails and
1 user used chatbot
1. Do Not Know How to Find Customer Service
1 user wanted to contact customer service for help because he was unable to find his application form but was confused about how to find customer service since there were no options for him to ask for help on this page.
Quick Win
1. Customer Service
Entry Point
If the users can communicate with customer service immediately on the page or show phone number at an obvious location, it could provide immediately help for those who are in need.
2. Users Felt Negatively Towards Additional Required Details
27% (4/15) users copied their reference number from email and pasted on field box, they did not understand why they had to fill out more information other than the reference number.
REC!
2. a> Split the Verified Reference Number and Telephone Number Page
In order to enhance user experience while tracking the application status, divide reference number and telephone number verification into two pages. The page would be much simpler, and the users only need to focus on one task at a time.
2. b> Log Enquire Applications into “Progress”
When users track a reference number once, log it into the “Progress” afterwards so that verification steps can be simplified for next time.
3. Unable to Go Back to Previous Page
53% (8/15) users would like to go back to application progress page to find their form again after entering “Enquire Another Application Progress”, which they found out that in order to do so they would have to tap on “Application” and then back to Progress.
REC!
3. An Entry Point Back to Previous Page
Tapping on “Enquire Another Application Progress” to slide up a layer on top, a ‘back’ button will be available on the new layer, which allows users to easily go back to previous screen to check on their applications.
Next Steps.
Prioritization
-
Put all usability test items all-together with Wishlist item, votes from members of different departments based on essential criteria
-
Prioritisation workshop could be considered, serval techniques will be applied like Kano model can help to determine each item’s value effectively
Product Roadmap
-
Define project scope and timeline based on resources and capacity
-
Build Now, Next, Future goals
Implement Fix
-
Collaboration between different departments to materialize new design based on roadmap