A Case Against Free Examination

Users are goal oriented, they complete tasks, perform actions and ultimately seek results."

The mistake of free examination

Free examination happens when users reviewing your product or design are asked to give open-ended feedback without instruction. As a designer, you've asked users to freely examine your work (I'm guilty of it myself). More often than not interface designers aren't properly prepared or educated on how to run usability tests. This is an important point because seeking measurable feedback during user testing is critical. Free examination produces less valuable feedback because users don't experience products this way organically.

Ask yourself; when would a user visit a website or an application just to consider the layout? Don't allow yourself to bias this question as the designer. The user isn't interested in your inspiration. To further this point, consider framing the question: "Why would my user visit our product/service?" Is it to ask themselves, "What are my thoughts on this design?" The answer is, probably not. Remember that users are goal-oriented, they complete tasks, perform actions and ultimately seek results. Users rarely visit products or applications just to experience a layout. No matter how much work we designers put into carefully crafting beautiful UI's, the aesthetics aren't the primary goal for the end-user. Requesting a free examination of our work comes naturally to designers because we are programmed to constantly seek creative feedback. This is a difficult habit to break.

Authentic Results

If you establish your objectives or user goals before beginning your product review/testing, you can avoid the pitfalls of free examination. However, be aware that in testing scenarios, participants routinely consider product functionality and design more than they would if they were outside of the testing environment. Some researchers blame this phenomenon on the Hawthorne Effect (more on that here). Regardless of the catalyst - the important takeaway is that the collected results must be more accurate. Think of it this way: A user experiences your product design much differently when they elect to interact with it vs when they are participating in a usability test. In the comfort of our homes, we are far less likely to study the interactions and patterns required to take action. Users have a goal, and any friction they encounter is dramatically more evident in the real world vs. during testing.

Lessons Learned

On one of our recent website projects, the user testing I conducted provided us with valuable results around an e-commerce ordering process - or so I assumed. The test was conducted with assumptions or expectations about what my findings would be. These assumptions were not revealed to the testers to avoid coaching responses (establishing hypothesis). What I learned after implementing my test findings was that real world users were still confused. During the testing phase, the participants made logical assumptions about the necessary steps needed to complete their actions (in this case: online food reservations and purchase). Once outside of the testing environment and with less context about the product and service they were interacting with, the experience broke down. The process wasn't as evident or clear to users after all. This information was at odds with the findings during the first round of testing.

Focus on Function

On one of our recent website projects, the user testing I conducted provided us with valuable results around an e-commerce ordering process - or so I assumed. The test was performed with assumptions or expectations about my findings. These assumptions should have been revealed to the testers to avoid coaching responses (establishing hypothesis). After implementing my test findings, I learned that real-world users still needed clarification. During the testing phase, the participants made logical assumptions about the necessary steps to complete their actions (in this case, online food reservations and purchases). Once outside of the testing environment and with less context about the product and service they were interacting with, the experience broke down. The process could have been more evident to users, after all. This information was at odds with the findings during the first round of testing.

Doing it solo

Guess what? We don't all have fancy UX departments. We can't all walk to our User Experience Team and ask for mind maps and user flows. Often, we designers are left to fend for ourselves when implementing things like user testing and persona development. We all could benefit from a top-down approach to UX in our organizations - yet the reality for many professional designers is often never that convenient. The good news is that smaller teams and freelancers can practice helpful exercises like avoiding free examination and implementing follow-up testing. We can have a more measurable impact on the product when we are ambitious enough.