What is the question and what can we test to answer it?
All of our tests begin with a question we’d like to answer. Some of the earliest tests we ran focused on pretty basic questions, but when they all came together they formed a backlog with two parts: What is the question and what can we test to answer it?
|Question||What can we test?|
|What’s the best way to design a country input dropdown?||A dropdown with different list orderings, fancy dropdowns with autocomplete, and different page locations for dropdowns.|
|Are multi-page forms better or worse than single page forms?||Test the same form with different page breakups: all one page, multiple pages, and a page for every element.|
It’s important to ensure that the question can actually be answered through this methodology. If you’re trying to answer a question regarding how users feel about a certain change or UI implementation, this probably isn’t the right methodology for you. However, questions concerning usability and user experience on a broader, more general scale are appropriate. These questions can apply to specific user interface elements like dropdowns, or an entire site-flow like a registration flow.