Singapore Airline Consultancy

Research focus

This project displays an Expert Evaluation our team performed as a design consultancy for Singapore Airlines.
The goals of this work were to evaluate Singapore Airlines mobile app against usability heuristics, design guidelines & principles and reviewer’s past experience, in order to find usability problems to prioritize and fix.

Methodology

Expert evaluation is a review of a product’s user interface by two or more usability specialists. The findings are assigned severity ratings and accompanied by actionable recommendations for improving the user experience of the product.

This type of study is effective to identify minor issues that would be difficult to observe with real participants (e.g. UI or brand guidelines inconsistencies). Also, larger issues that go against best design practices would get caught easily, so they could be addressed before wasting any time with the setting and recruitment for a typical usability test.

PHASE 1 • Qualitative evaluation

5 expert designer of our team tested Singapore Airlines App against two major competitors: Emirates and American Airline. For each app tested we have compared 4 key touchpoints:

• Onboarding screens
• Homescreen
• Flight search
• Fare selection for a return flight

PHASE 2 • Scorecards

In order to complement the qualitative data with tangible usability KPIs, we decided to benchmark the different apps by creating Design Scorecards.

Each design expert participated in a survey designed to extrapolate key metrics such as App Usability, Appearance, Satisfaction and Loyalty for each mobile app tested. This survey has been integrated with standardized measurement tools such as:

• SUPR-Q – Standardized User Experience Percentile Rank Questionnaire
• SUS – System Usability Scale
• CSAT – Customer Satisfaction Score

PHASE 3 • Prioritization

Phase 1 gave us more than 350 reviews and over 7 hours of recorded observations. We distilled feedback into 39 recurring themes and asked each expert to rank their gravity following these criteria:

• Cosmetic/Content problem: doesn’t need to be fixed unless extra time is available
• Minor usability problem: fixing this should be given low priority in product roadmap
• Major usability problem: important to fix, should be given high priority in product roadmap
• Usability catastrophe: imperative to fix this as soon as possible

Deliverables

Results have been summarized in a slide deck shared with the client. This document focuses attention on strengths and weaknesses of the Singapore airline app comparing the qualitative and quantitative data with the two major competitors.