User Testing v/s Usability Testing

This research is powered by CanvasFlip. x

Who- Designers, Product Managers

User Testing v/s Usability Testing - With an UBER example

My dear friend (ahem, I wish) Travis, now Founder and CEO of UBER App got stuck in a traffic in Paris on a rainy day and couldn’t get a cab. He took the problem in hand and started going to friends’ / bus stops / taxi stand / building exits, asking them if they would like to have an app which could get them a cab in a click (and how). That’s user testing. It is more like testing the utility of your idea or app. It involves understanding how people do this thing as of today and will (and how) they use an alternative to do that same job. Typical result of user testing is user persona of your target user segment.

Now there were enough friends’ and people telling Travis that they’d love to get a cab right from their mobile app. So he got an early design and ask them to try booking a cab ride. He observed and identified where people were getting stuck in booking a ride. Fixed it. Showed them again. That’s usability Testing. It helps you to understand if people can effectively use your app or not, where are they getting stuck and what should you do so that they don’t’ get stuck.

( Screenshot from one of the earliest version of Uber know as Uber Cab )

And that’s the same story behind every successful app - Tinder, AirBnB, Snapchat, you name it. The difference between user testing and usability testing is really simple - User Testing is about “Will this user use my product?” where usability testing is to figure out “Can this user use this product?”. People often confuse these with each other. Once you understand these terms better, you can use these methods at right time and get better input for better user experience.

Let dig deeper and see how and when user testing and usability testing add value in creating good experience. To keep things simple, we can understand differences in terms of the stage they are required, Expectation out of it, Questions asked and methods/tools being used.

When should you do:
- User Testing: Right after you have got the idea.
- Usability Testing: As soon as you have drawn those random hand drawn sketches on paper

What to expect from:
- User Testing: Do people need the solution I am working on?
- Usability Testing: Can people easily use the app ( for a given set of task) I am working on?

How should you run an effective:
- User Testing: coffee shop discussion, in-person discussion at friends’ place, office, club, bar, Virtually at Skype, Google hangout, google forms, Type form, Survey Monkey.
- Usability Testing:
  - You can create a rapid prototype on CanvasFlip within 5 minutes. You can use your hand drawn sketches.
  - Add small task for users.
  - Go to your users and ask them to do the task on the prototype
  - On CanvasFlip, you can also remotely record how users are navigating through to complete the task.
  - Instantly observe conversion funnel and heatmap on CanvasFlip to see where users are getting stuck in your flow and review videos on how to improve.

What to ask during:
- User Testing:
  - How do you currently do <problem you are solving>?
  - Did you even think of a better way of doing this?
  - Will you like to do this task <the way your solution works>?
  - Will you pay money (for transaction apps) / share content for this solution?
- Usability Testing:
  - Can you try doing this <new way of solution>?
  - How would you like to login to this solution?
  - Can you get <a small task in your solution> done in 10 seconds?

Inspire

(An example of Usability testing results on CanvasFlip - See More )

TL;DR: User Testing is testing your users whereas, Usability Testing is testing the interface.

Now you know how user testing differs from usability testing, how do they add value and when. Next time, when Jonathan comes to you from his swanky cabin asking you to do a user testing - help him understand if he really wants to do a User Testing OR usability testing. You may as well save his and your time in random internet research, choose the right tool in achieving the results he is looking for while you look cool.. 😉

All the best with your next user testing (or usability testing)!!

CleverUserName

This article is completely false and will confuse anyone that reads it. User testing is not AT ALL “will users use my app?”. That is called USER RESEARCH and in an ideal world happens before any/much investment is made to build an MVP. ‘User testing’ and ‘usability testing’ are terms that are used interchangeably and both refer to testing in which you observe a user use your app/website/interface whether in a rough state or a shipped state.
- Vipul
  
  User research has multiple component. User testing is a part of it and let me help you understand that is is NOT same as user research.
  
  User research helps you understand the traditional solution to your problem, market research, feasibility testing and many other things whereas in user testing, you just got an idea of alternate way to doing thing and you go to your friends, peers and ask them how do they like it.
  
  Testing and observing “App/website/interface” is a very small part of entire process. For example: for an e-commerce app, if users get a better discount, he will also opt for the max discount option. As a UX designer, it is to know and understand, what inspires my user most and how to emphasize it. That’s User Testing!!
  - CleverUserName
    
    That is all information you find out in user INTERVIEWS and user RESEARCH. Not user/usability testing.
Dean Birkett

You don’t test ‘users’, you test your product. Testing a ‘user’ is testing them as individuals…. you would perform user research to get insights from individuals (but not to test them), you would perform usability tests to test the product or idea.

Will you like to do this task ?
Will you pay money (for transaction apps) / share content for this solution?

Are terrible questions to ask at any level. The answer is “Yes!!! It’s brilliant, great job!” - it tells you absolutely nothing except that the test participant want to tell you what you want to hear. Ask the five W’s (and How) questions, and avoid yes/no questions at all costs.
- Vipul
  
  Good point!! you DO NOT test users you test users. You still test your idea with users and if (or how) users will user the idea.
  
  About the questions, it is to spark ideas for the readers not to use it as it is. Also, you missed the use-case, it’s about about the getting a Yes/NO but it’s more like my example, “Will you like if you can get a cab from your mobile app?” Now, it has a simple answer “Yes!” but in most cases you will figure out that users are comfortable with current means and won’t shift without significant incentive.
  
  The five W’s are helpful at the very very beginning on the process ( even before you have drafted the solution )
  - Dean Birkett
    
    I’m sorry but that’s just not how you perform user research. If the goal is to uncover the need then ask about their process, “How do you currently signal a cab?” opens the door to discovery and opportunity.
    - Vipul
      
      I’ll have to respectfully disagree. Going with your use case, tell me this - Do you get an UBER same you ‘signal a cab’? However “when do you call a cab - 5 minutes before start, after I left for the place” Or “how often do you call a cab in a week” will add more insights.
      
      Will try to sleep over with your views for a while and revert if I see an overlap of thoughts. Thanks for sharing your views.
      - Dean Birkett
        
        Note the key part of the sentence, “opens the door to discovery and opportunity”… you form questions around the hypotheses that you have. You don’t perform user research without having an idea of the problem that you are wanting to tackle.
        
        If you (as a business) have defined a problem with how cabs are ordered (flagging them down, or by telephone), then in a discovery workshop you may have some suggestions of what routes you *could* take to solve the hypothetical users problem.
        
        If the user stories are “I as a passenger want a quick way to order a taxi, without having to pick up the phone”, and “I as a passenger want to pay my fare without having to exchange money physically”.
        
        You form questions around these to try and see where the matches are…. you’ll uncover new insights along the way… perhaps one of those insights was “I want to feel secure by knowing who my cab driver is before I get into the cab”… that uncovers more opportunities, and more discussion.
        
        What you don’t do is say “Hey! I’ve got an idea for an app where you can order a taxi without picking the phone up, great isn’t it?”…. fortunately neither did Uber.