Remote user testing sites reviewed

On a recent project, my client needed a quick way of gathering high-level user feedback on several first-iteration wireframes. We were after general stuff like “Where would you click if you were looking for…?” I suggested we try remote user testing for three reasons:

1. It’s quick
2. It’s (relatively) inexpensive
3. It’s (surprise surprise) remote.

I have carried out remote user testing before but there are now a lot more companies offering online tools, so I thought it would be a good idea to review what’s on the market.

Here’s a quick guide to what I found out about the three products I looked at:

• Usabilla: Test interaction with wireframes and conceptual designs
• Loop11: Recruit users and run surveys and user tests on live websites
• UsabilityHub: Test first impressions and interaction with wireframes and conceptual designs.

Usabilla

My verdict: Pretty basic testing but it’s easy to set up and use straight away and the pricing model is flexible enough for everyone.

What does it do?

At its most basic, you can upload an image (or take a screen grab of a website), ask users a specific question about what they see and get them to click and add comments. User clicks are recorded and displayed on a heatmap along with the any comments and the time the user took to click something.

You can invite participants by sending them a direct link, share your test via Twitter, LinkedIn and Facebook or add an invitation widget to your own website.

If you pay for customisation, they say you can also add online surveys and personalised entry and exit pages (although I haven’t tested this).

What does it cost?

• 30 day free trial on all plans
• $19 per month = 1 active test, 20 participants, no customisation
• $49 per month = 1 active test, 20 participants, with customisation
• $89 per month = 3 active test, 100 participants, with customisation
• $199 per month = 10 active test, 120 participants, with customisation
• 25% discount for 1 year sign up and %50 discount for 2 year sign up.

Loop11

My verdict: Pretty comprehensive but quite a big up-front cost.

What does it do?

Setting up tests is really easy via their online dashboard. You can  create multiple choice, Likert scales and open-ended questions and set tasks for users to complete when viewing specific web pages on live websites. Test users are guided through your questions and tasks via a permanent header added to the chosen website.

You can invite your own participants by sending them a direct link or add an invitation to your site. You can also recruit people via Ethnio or “Buy” participants from Loop11′s own Online Panel.

The reporting dashboard contains an impressive array of tables, pie charts and bar graphs showing user data and responses to questions. User interaction on tasks is illustrated with click stream diagrams and “Participant Paths” but there is no heat map showing exactly where  they clicked on each page (at least not that I could find).

What does it cost?

• $350 per project
• $1,900 per year for businesses with less than 10 employees, not-for-profit organisations, educational institutions and government
• $4,900 per year for businesses with less than 100 employees
• $9,900 per year for everyone else.

UsabilityHub

My verdict: “Karma” rewards mean you can set up tests for free but it’s difficult to know which of their three products is right for the job.

What does it do?

UsbilityHub is really three overlapping products, each doing something slightly different.

With Fivesecondtest you can set up a timed test where users are shown a webpage design before being asked a series of open-ended questions about what they see. Keywords used most frequently by participants are shown graphically and all responses are available as raw data.

With Navflow you can set up a stepped journey using images to learn how users might navigate your intended website. Conversion funnels illustrate how participants got on (e.g. success and failure percentages).

With Clicktest you can upload a single webpage design and ask users where they would click if they wanted to complete a specific task. User clicks and times are recorded and dislayed on a heatmap.

What does it cost?

• Gain reward points to spend on tests or pay as you go
• $20 per month = 100 user response per month
• $50 per month = 250 user response per month
• $100 per month = 500 response per month
• $200 per month = 1000 response per month.

And there’s more…

Each product has its pros and cons. In the end I used Usabilla because it was the best fit with the project but I can easily see the need to use one of the others on a different project.

And there are lots more products too. Here’s four more I found but didn’t review in detail:

• Intuition HQ
  Test first impressions and preferences on design variants via desktop and iPad.
  Prices start from $49 per month
• Userzoom
  Run surveys and card sorting, and test interactions on live websites and mobile interfaces
  Prices starting $1,000 per month
• Usertesing
  Recruit users and test interaction via desktop and mobile
  Prices start form $39 per participant
• Chalkmark
  Run surveys and test interaction with wireframes and design concepts
  Prices start from $109 for 300 days.

And if you’re hungry for more, Matt Milosavljevic has written a great article on UX booth about Picking the Right Tool for your Remote User Testing.

Posted by craigbrewster on April 25, 2012

http://yourmumux.wordpress.com/2012/04/25/remote-user-testing-sites-reviewed/