Usability/User Testing

Facebook
Twitter
LinkedIn
Engineering Area
Assessment & evaluation
Group or Individual
Group
Amount of People
Up to 15
Type of Class
Practice-based
Duration of Activity
Between half an hour and one hour
Type of Activity
Collaborative team setting

Description

What is this technique about

Usability testing is a popular research methodology in user-centred design. It is also commonly referred to as user testing and means testing with users (not to be confused with “testing users”). In a usability testing session, a researcher, also known as a facilitator, observes participants as they interact with one or more user interfaces while completing specific tasks. The researcher seeks to understand the participant’s behaviour and feedback to identify problems in the design and uncover opportunities for improvement:

    • How well does the solution idea fit the users’ need or problem?
    • Identifying problems in the design of the product or service
    • Uncovering opportunities to improve
    • Learning about the target user’s behaviour and preferences
    • To which extend are performance or quality objectives reached?

In early phases of the creative process, the user interfaces are typically sketched or prototyped to simulate the future behaviour of the solution. In later phases a ready implemented solution can be used.

During the early stages of the creative process, user interfaces are typically sketched or prototyped to simulate the future behaviour of the solution. In later phases, a fully implemented solution can be used. Evaluations can be formative or summative in nature. Formative evaluations aim to “form” the design and to improve the performance and quality aspects of the solution during the design and development phases. They involve evaluating the product or service iteratively and identifying usability problems that can be eliminated to refine the development specifications. The target audience for the observations and recommendations is the project team itself.

In contrast, summative evaluations aim to determine if the complete or near-complete design meets specific measurable performance, quality, and/or satisfaction goals, establish a usability benchmark, or make comparisons between variants. Summative evaluations are typically conducted under realistic conditions. They provide feedback on the effectiveness and efficiency of the design and help to establish its usability.

Where does it come from 

Usability testing has its roots in Human Factors Engineering, which is a field that emerged during World War II to address the need for safer and more efficient military systems. In the 1960s and 1970s, researchers began to apply Human Factors principles to the design of computer systems, leading to the development of usability testing methods.

The first formal usability testing method is often attributed to a psychologist named John E. Karlin, who was part of the team that developed the first touch-tone telephone keypad in the 1960s. Karlin used usability testing to evaluate the design of the keypad and ensure that it was intuitive and easy to use.

Since then, usability testing has become an essential part of user-centered design methodologies and is widely used to evaluate a variety of products and services, including software, websites, mobile applications, and consumer products. were 

For which purposes it is used (why in your engineering teaching)

Although professional designer may be skilled at designing a solution, usability testing is still crucial to achieve a great user experience. Even the most talented designers cannot create a perfect or even a good enough user experience without an iterative design approach based on real user observations and interactions with the design. Designing a modern solution involves numerous variables, and the human brain introduces even more variables. With countless combinations possible, the only way to ensure that design is efficient and effective is through testing.

How to use it

Usability testing can take many forms, but it typically involves a facilitator, tasks, and participants. The facilitator guides the participant through realistic tasks while observing their behavior and listening to their feedback. 

The tasks are designed to represent activities that users might perform in real-life scenarios. They can range from specific to open-ended, depending on the type of testing and the research questions being explored. For instance, if you’re revamping an e-commerce website, you can evaluate the user-friendliness and overall satisfaction of the updated purchase process by assigning tasks that involve observing how participants navigate the site, search for specific items, add them to their cart, provide credit card information, enter address details, and so on. It’s crucial to be precise in task wording since small phrasing errors can cause misunderstandings or influence participant behavior (known as priming). Tasks can be delivered verbally or handed to participants in written form.

The participants should represent the target user group, either by already using the product or service or sharing the same background or needs as the target user. 

During usability testing, participants are frequently encouraged to verbalize their thought process (known as the “think-aloud method”). The facilitator may prompt participants to describe their actions and thought patterns as they carry out tasks. The aim of this technique is to gain insights into the participants’ behaviors, objectives, thought processes, and motivations.

How to implement this techniques online

Preparation, what do before the session
    1. Split your class into groups of 4 – 6 participants.
    2. Make sure that each group has a stable video conference tool with a good bandwidth ready, since screen-sharing will be needed for observing the interaction between test-users and a digital product or service prototype. In case of a physical solution, the camera will be needed.
    3. Make sure that each group has their sketches, prototypes or implemented solution ready for remote testing.
    4. Prepare a digital whiteboard for each group for analysing the test data.
    5. Define the research question and determine what you want to achieve through the usability test. What do you want to learn about your product or service and its users?
    6. Identify the user groups that you want to target for the usability test. It’s important to ensure that your participants are representative of your target users.
    7. Develop a set of test scenarios that reflect typical user tasks or goals. These scenarios should be specific, realistic, and relevant to the users you are testing with.
    8. Recruit participants who are representative of your target user groups. Participants should be screened to ensure that they meet your criteria for participation.

You can also let each group define their own research question, develop test scenarios or recruit participants.

During application, i.e., while giving the session

    1. Introduce the method and its purpose.
    2. Let the students conduct the usability test by having participants complete the test scenarios while being observed. The researcher should take notes on participant behaviour and ask follow-up questions to understand their experience.
    3. Let the students analyse the data collected during the test to identify patterns, trends, and issues that emerged during the test.

Follow-up, about what to do after the session

    1. Wrap up the exercise, e.g. let each group or team present their results.
    2. Ask each group, what they find important for the next steps in the creative process.
    3. Let the students create a report that summarizes the findings from the test and provides recommendations for improving the product or service based on those findings.
    4. Let the students use the findings and recommendations from the usability test to make improvements to the product or service, and then conduct additional usability tests to verify that those improvements are effective.
    5. Store the results.

Examples and/or testimonials

Below is an example test script from the course “User Experience Design” at Hochschule der Medien of Prof. Dr. Christoph Kunz. A team of students had the task to design an application, which supports the advertisement and application for tiny jobs in a neighbourhood. 

Instructions:

    • “prepare” prototype. 
    • Short presentation of the idea

Usability test: Clarify whether a recording (video & sound) is OK! (Serves only for transcription. Recording will be deleted and not handed in).

Tasks for testing:

    • Log in with the stored login data.
    • We are now going to the CareWoche homepage. Please view the homepage without clicking on anything. What is your first impression?
    • Try to create an empty job.
    • Try to find the job “Walking Kaja”, look at the provider and accept the job?
    • Where can I see which jobs I have already posted/accepted?

Post-test questions:

    • Please name 3 positive aspects
    • Please name 3 negative aspects that need to be improved.
    • What image did you form during use?
    • Do you have any suggestions about XY?
    • How do you feel about the navigation?

Tools needed

You will need a platform to share screens and communicate with the participants, such as: MS Teams, Zoom or similar.
You will also need a white board solution with digital sticky notes such as Miro or Mural:

    • Zoom
    • MS Teams
    • Miro
    • Mural
    • Timer (phone, watch, computer)

Resources

Links

Moran, K. (2019, December 01). Usability Testing 101. Nielsen Norman Group. Available at: https://www.nngroup.com/articles/usability-testing-101/

Videos

Blink. (2012, August 14). Usability Testing of Fruit. [Video]. YouTube. Available at: https://youtu.be/3Qg80qTfzgU

Papers

Rubin, J. and Chisnell, D. (2008): Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley. ISBN: 0470185481.