Kalyani Katariya
UI/UX Designer and Researcher
Made with

Usability Study of the Vision 'QuickCheck' App

Vision screening made quick and easy!

Overview

Designed vision screening app for Washington state school nurses reducing the screening time by 4 minutes. Android app was tested with 4 school nurses and 1 doctor. Used an agile methodology and design thinking principles to design and develop the app. 

Goal

The goal of the project was to help school nurses to conduct vision screening in schools in Washington state faster and efficiently.

Design Question

How might we help Washington state school nurses to identify vision disorders?

Solution
  • Implemented end to end research, design, testing and test analysis of the quick check app using Figma.
  • Provided recommendations and future development plans for the current design team.
  • Developed a framework for using the mobile device as a clinical assessment tool. 

Impact
  • Vision screening with the old solution was taking ~10 minutes, with the new design, I was able to reduce the screening time by 4 minutes.

  • Published a usability framework for using the mobile device as a clinical assessment tool in ophthalmology which can be used by the wider health app developer community in the future.
  • The project received Funding from SEVA Foundation ($125,000) to develop a mobile van carrying vision screening tools. QuickCheck app is going to be part of this screening toolset.

My Role: UI/UX Designer (Team of1 UX designer and 5 developers)

Tools: Figma, draw.io, google form

Timeline: Jun 2018 -Jun 2019

Client: Educating Young Eyes (EYE)

Publication Link 

Process
QuickCheck App
Research
QuickCheck App
QuickCheck App

Persona

During user research, I learned that school nurses, teachers, volunteers, and parents are our different personas. EYE Center research group at UW wanted to address the challenge of school nurses by providing the more accurate, relevant and speedy process for vision screening. We referred to them throughout the product development process. We created Ad hoc persona for the QuickCheck app.

Screenshots from the old web-based application: Near Vision Toolbox

Usability concerns for the old solution​


  1. The old solution was developed as a web application and the current need was for a mobile-based solution.

  2. Layout, typography, the color was out of date and did not provide clear navigation for the users.

  3. The time required to finish all the screening tasks was 10 minutes.

  4. No clear and concise instructions to conduct the screening.

  5. EYE center research group has previously developed vision screening applications as shown in the below screenshots.
Insights
QuickCheck App

Usability considerations for a new solution

From the above design concerns, the research group identified the following solution

  1. Design a mobile-based solution.

  2. How can we use mobile as a clinical assessment tool, that can be efficiently used for long hours and for screening more than 100 kids?

  3. Design kids friendly screens.

  4. Propose layout, typography, logo for the new solution.

  5. Customizing app features based on user's habits of using the mobile phone.

QuickCheck App

Scenario

Since we were redesigning the vision screening tool for a mobile device it was important to know how the user might use the app in a real situation. Optometrist Dr. Alan Pearson have experience of performing vision screening and working with the school nurses/ teacher. He helped us to imagine the scenario for QuickCheck. We choose the school nurse persona to describe the vision screening app scenario. 

The scenario helped to clarify the user’s goal, a process they might use and inputs to the system.


QuickCheck App

Vision QuickCheck app flow diagram

Based on the scenario I came up with a possible high-level flow of the QuickCheck application. Flow helped to understand the user’s interaction with the QuickCheck app at various stages.

Design

Sketches

I usually start the design process with low fidelity wireframes. It allows me to iterate through many design options quickly.  I created the paper wireframes for the QuickCheck app and brainstormed these designs with my research group. I created two versions based on a discussion with the team. The below screenshots shows the second version of the paper design.

Mockup

I used Figma to design the mockup. There were 2 iterations on the mockup.  It helped me to identify the key user flow of the system that will enhance the user experience such as -

  • Auto-populating the student record from the database
  • Providing error messages and confirmation messages after each test. 
  • Using the big size buttons to support different thumb sizes. (Balakrishnan, 2008).
  • Using different colors for each test to indicate the transition between the tests.
  • Customizing the test screens to record the kid's answers based on the left-handedness and right-handedness habit of nurses.
  • Allowing the user to retake the test.

QuickCheck App

Typography

Roboto is a sans-serif font that was specially designed by Google for Android mobile phones and tablets. I decided to use the ‘Roboto’ font it also helped to achieve the readability of the text. (Sharrock, 2018)   

QuickCheck App

Color

Green is restful for eyes and is also associated with healthy living. (Soegaard, 2019). Hence, a green color palette is fitting for an app that promotes eye health. I used green color for the primary app screens Along with that, I used grey for the text and included a white to give a calm and clean appearance.

QuickCheck App

Participatory design with the UW Kid’s Team for QuickCheck

The UW Kids team research group at the University of Washington includes kids ages 7 to 11, software designers and Ux researchers. Their work is to suggest and evaluate software design, share their experiences with various technologies, web applications, mobile applications using the participatory design method. From my experience with the Kids Team, kids are full of creative design ideas and they are not afraid of sharing their feedback.

While working on the QuickCheck app I learned that kids are also one of the users of the app. Though they are not literally operating the app, they are experiencing the entire app while they are getting screened by the school nurse.

The proposed design of the QuickCheck app also intended to have a kid’s friendly mode (i.e. make the screening more interactive and display minimalistic content on screen). Hence, I decided to conduct the co-design session with Kids Team for the QuickCheck app.

Design method: Sticky Notes

Research question: Kids’ experience with the app and design ideas to improve the app.

Design dislikes

1.CISS Survey questions were boring.

2. The HOTV letters on the same screen are distracting.​

Design likes

1. Kids thought the app is simple and easy to use.
2. Tests are quick and easy to follow.

Suggestions to implement for the next design iteration

1. The team decided to implement the below suggestions in the next design cycle

2. Make the CISS survey more interesting using more interactive mode maybe by adding some music.

3.AB testing to determine the HOTV letters position on the screen. The screenshots in the below section show the likes and dislike shared by kids during the session.

Final Solution
Usability test


Usability test
The next stage was to conduct the usability test. The usability test helped to identify usability deficiencies at an early stage of product development. It helped to address the design problems and provided considerations to enhance the user experience.
You can read all the details stages and descriptions in the thesis here. All the documents generated as a part of the usability test are in the appendix section of the thesis.

Participants demographics
Participants group was diverse, out of 5 participants 3 were nurses, 1 was a doctor and one was a technical expert. None of them have ever used a mobile device as a clinical assessment tool. Except for technical expert, all the other participants were familiar with the vision screening for kids in school.

QuickCheck App

Identifying the primary task

The following tasks were identified for the usability test. In order to understand usability, the critical functions of the app were chosen as tasks. For each task, a scenario was narrated to the user.

Usability Metrics

Usability Metrics

Constructing metrics and measurement framework

From literature and discussion with the research group I identified Efficiency, Effectiveness, Learnability, Operability is the important metrics for the QuickCheck app.

Effectiveness: To measure error occurred and the task completion success rates.

Efficiency: to measure the time required to complete the vision screening test.

Learnability: Assess the learnability of the app’s functionality among first-time users and whether it improves over multiple usages of the app.

Operability: Identify the operability challenges related to the handling of the mobile device during the vision screening

QuickCheck App

GQM framework to measure usability metrics

To define the measurement framework Goal Question Metric model was very useful. I used the Goal Question Metric (GQM) model, identified questions that provide the basis for quantitative and qualitative metrics for the usability study. The usability metrics and framework proposed here can be used by any other mHealth Ophthalmology applications in future

QuickCheck App

Understanding the effectiveness

Usability test moderators recorded the participant’s ability to complete the tasks without prompting along with errors occurred during the test. Errors and completion rate suggested that there is a design issue with calibration and auto-populating the records tasks functionality.

Common errors that were reported during the usability test:

  • Accidentally clicking on skip test
  • Clicking the retake test instead of the next student.
  • Calibrate self instead of the user
  • Entering today’s date instead of the date of birth
  • Record the correct answer instead of students answer

QuickCheck App

Understanding efficiency

Time on task is the metric used for measuring the efficiency of the QuickCheck app. The time required to conduct the old test was ~10 minutes.

For all the tests, the minimum time was recorded during the third attempt of the vision screening test i.e.6 minutes which indicates that the efficiency of the app was improved than the old design by 4 minutes.

The table shows the average time taken by each test.

QuickCheck App

Understanding learnability

The graph shows the min and max time required to conduct each test over the 3 attempts. For CISS there is no significant variation as compared to the other three tests. This observation suggests that the CISS test learning curve is stable over a period of time. Not much learning is required to conduct the CISS test. Response to one of a question in SECTION B (Functionality) of MARS test – user’s a learning experience with the QuickCheck app.

QuickCheck App

Understanding operability

Another usability metric that we gather more understanding about was the operability i.e. the ease of the mobile handling during the vision screening test and customization options in the app to suit the user's habits such as left-handedness or right-handedness. The response confirmed that 4 out of 5 participants think that it has been addressed by the design.

QuickCheck App

Results of the interview

The five interviews were analyzed for the themes across participants to identify opportunities, needs and goals for the end-user. Three themes seem to have emerged:

1. Motivation to use the app

2. Barriers in the app usage

3. Factors that might influence the app usage. 

Mobile App Rating Scale(MARS) Test Result

Mobile App Rating Scale(MARS) Test Result

Post- Test Questionnaire (MARS Test) Results for Overall Metrics

After task completion, a post-test questionnaire was given to participants to rate the app for the overall four measures. The mean rating demonstrates the quality and desirability on various aspects such as engagement, functionality related to the app. It gives them insight into which areas of the app can be focused during the next design iteration. Mean rating suggests that most ratings hovered around three, four or five stars. This indicates the overall quality of the app is good with scope for further improvements. 

Reflection
QuickCheck App

Learning

I learned how to manifest an idea into reality, children's vision issues and their impacts on children’s learning ability and communicating with cross-functional team members. While performing a usability test, I saw the hurdles that users were going through for some of the tasks in the app, I developed empathy towards users.

Great job…….Keep truckin’.
- Director of Functional/Neuro Optometry