Wizard of Oz Studies in UX Research

What Is a Wizard of Oz Study?

Wizard of Oz (WoZ) study is a UX study that allows users to think that the system is functioning normally while a person operates the system from behind the scenes.

The term Wizard of Oz comes from the story of the Wizard of Oz, where the wizard is powerful but just a person hiding behind the curtain.

The WoZ study allows the researcher to test an idea without the expense of developing the application.

The study of the Wizard of Oz is particularly applicable when conducting tests of:

AI Systems

Voice Interfaces

Chatbots

Smart Assistants

Complex Automation Tools

One does not need to develop the system, but instead, simulate the system manually. This is particularly useful, in my opinion, as it helps teams avoid developing something that users may not need.

How to conduct a Wizard of Oz study step by step in a free-form manner

Define what you want to accomplish
– Determine what you want to investigate. For example, what are people’s responses to voice commands when it comes to a beauty product recommendation system.
Develop a basic prototype
– Develop a basic front-end to make it look and feel legitimate to users.
Choose a “Wizard”
– Assign a researcher to be the one controlling responses to users.
Execute the study
– Allow users to interact with it without them knowing a human is controlling responses.
Observe user responses
– Observe user responses to what they are doing. What are users’ confusions? What are users’ expectations?

Tools Used for Wizard of Oz Studies

Figma for creating clickable prototypes
Zoom and other screen sharing tools
Google Forms for controlling manual input
Chat simulators
OBS Studio for recording sessions
The most important tool isn’t necessarily any of these pieces of software. It’s the controlled simulation of how the system will work.

Research Studies Using Wizard of Oz Method

Study 1: Voice Assistant Interaction (Kelley, 1984)

One of the initial experiments with the Wizard of Oz method was on natural language interfaces. In the study, users interacted with the system, believing it to be automated, while the researcher interpreted the user’s command privately and responded manually. The study revealed the natural way users give verbal instructions.

Study 2: Smart Home Interfaces

Research on smart home interfaces often utilizes the Wizard of Oz method to simulate smart home systems. The study revealed that users expect contextual awareness and verbal responses, which go beyond the current scope of artificial intelligence.

Study 3: Healthcare Chatbots

The Wizard of Oz method was used to study healthcare interfaces to determine user interaction prior to artificial intelligence implementation. The study revealed that users trust the interface based on the empathetic response of the system, along with the response time.

Strengths and Limitations

Strengths

Inexpensive to create
Allows ideas to be checked early on
Easy to modify

Limitations

Ethically challenging since debriefing’s required
Not scalable
Realism’s not as high as a fully created system

Conclusion

Wizard of Oz studies are an important UX method used to investigate intricate interactions without having to create a fully functional system. Here, human participants are used to mimic a system’s functionality. This provides an early indication of what users expect, how users will behave, and what limits feasibility.

It’s an important method to guide design choices quickly while avoiding untested tech.