Why do we study usability? The answer is simple - to save MONEY. Usability testing assesses users’ ability to use the site features efficiently, and carry critical site tasks without encountering significant obstacles.
Obstacles = Time = MONEY
We study usability because we like to be efficient. We like to make things work better and in doing so, it helps save MONEY! If website users are struggling with the workflow on a site, they tend to waste time doing things over and over again...this is frustrating for the user, and guess what? It wastes MONEY! (Are you seeing a theme here?) Improving the usability of a site reduces abandonment and increases conversion and retention rates...have I got your attention now? Let’s learn a little bit more about the process of a usability study… Check out the user testing resources at the bottom of this post!
Why Test Administrators
The purpose of user testing is to assess users ability to use the site features efficiently and carry out mission critical site tasks without encountering significant barriers… but what happens when the site’s power-users are staff and administrators?
If your organization uses a CMS simply because it is easy to update site content, know that there are a lot of other administrative efficiencies a Drupal site can accomplish for an organization - if it’s easy to use.
Often sites are designed for the end user but leave the user experience of the administrator as an afterthought, or even forgotten altogether.
User testing the administration of your Drupal site is a solid investment, as increased efficiency means less time fiddling with webby things and more time to pour into your organization’s mission!
The Purpose: Why are we studying the usability?
There are three things that I always investigate, prior to beginning a usability study. Website Type, Organizational Focus, and Organizational Site Admin Style. Essentially, I look at the ‘What’ (site), the ‘Why’ (goals/objectives), and the ‘Who’ (stakeholders, content creators, admin).
The ‘What’ website type
- Brand Presence/Brochure
- Marketing Campaign
- Content Source or Resource Library
- Task-Based Application
- E-Commerce
- E-Learning Platform
- Social Network/Community
- Web Portal
- Intranet
The ‘Why’ organizational focus
- Products
- Brand
- Programs
- Courses
- Services
- Resources
- Educational
- People (brand)
- People (community)
- Portal
The ‘Who’ organizational site admin style
- Single Administrator / Webmaster
- Senior Editor / Junior Content Creators
- Sectional Content Managers (department, store, content type)
- Complex Access Restrictions (by taxonomy, field-level, etc)
- Workflow Restricted Access Rules
- Crowdsourcing and public submission moderation
The Users: Identifying roles and groups
Once I understand the purpose of the site, I need to understand who is involved with the site. It’s not as simple as talking the primary stakeholders, as there are often many levels of involvement in any site.
Users: Roles and Permissions
- Policy & Decision Makers - access all/none depending on organization
- Subject Matter Experts - where are they in the content creation workflow?
- Content Moderators - voice, tone, grammar, facts, formatting
- Content Creators - how much control do they have over content?
- Content Custodians - freshen, cleanup: redundant, outdated
- Content Consumers - contributions: commenting, flagging, favoriting?
Users: Potential Study Stakeholders / Contributors
- Marketing / Sales
- Copywriter / Content Editor
- eCommerce Store Admin and Order Fulfillment Staff
- Faculty / Educational Consultants
- Board Members / Executive
- Project Managers / IT Team
- Department Staff
- Customer Service / Help Desk
- In House Designer
- Outside Vendors
- Volunteers, Vendors, Suppliers
Users: Common Stumbling Blocks
When considering users, it is also important to address some of the common stumbling blocks that these groups may encounter - what makes sense to the content creator that is well versed in the system / product, may not make sense to the site user that is interacting with the site for the first time.
- Conflicting vocabularies
- Inexperience with CMS model
- Steep learning curve
- Organizational shift
- Unfinished site / content
- Subjective ideas
Satisfaction Is All About Comparisons a note about feedback
Users will be comparing a system / site / product with every experience they’ve had in the past - it’s important not to overlook what a user’s expectations are, as these expectations are what shape and determine a user’s satisfaction.
- An experience compared to nothing… is indifference
- An experience compared to expectations… is better/worse/different
- An experience compared to needs… will be met/not met/ a mix
- An experience compared to excellence/the ideal product… is good/poor quality/good enough
The Planning: How to organize and carry out a usability study
Once we have a clear understanding of what and who we are studying, we need to execute the usability study. But where do we start? First, we need to assess the user segments, and note the processes that they will be responsible for. With this information, we are able to look at these user tasks on a granular level, determining what each user is responsible for creating and managing. All of this research and assessment, leads to the creation of user scenarios - who are you studying, what would they like to do, and why are they doing it.
- Purpose of site
- User segments/groups
- Purpose of study
- Goal of study
- Suspected Issues
- Contact List
Planning: identifying users’ processes
It is important to gain a clear understanding of the processes that each user group will be responsible for - in most cases, each user group will have unique processes and goals.
- Product configuration
- Tool setup, or installation
- Signup & registration
- Store checkout administration
- Booking resources
- Managing submissions
Planning: identify how users create or modify content
Understanding how users actually perform the processes of the site is crucial to understanding their potential challenges - it won’t be helpful to note that the user couldn’t perform a task, if you didn’t uncover the point at which the workflow was broken.
- Curating, tagging, categorizing
- Revising / editing content
- Formatting / styling
- Connecting related content
- Submitting content for moderation
- Publishing / archiving
Planning: the 7 ‘C’s of content management
- Conception - who decides when content is needed?
- Creation - who writes, adds and formats content? What kind of media attachments are included?
- Categorization - how is it organized?
- Confirmed - publishing workflow - does content need an approval process?
- Changes - is there revision-tracking? how does it work? what content will need to be removed or archived?
- Community - is there social sharing?
- Consumption - anything special about how the content is consumed (access control, notifications, special analytics or tracking, feeds, subscriptions, etc.)
Planning: identifying content workflow inefficiencies
- 7 Cs
- Types of Content
- Factual Updates
- Notifications
- Vetting
- Batch Add
- Clone
- Image Sourcing / Editing
- Multimedia
- Content Reuse
- Context and Connections
- Editorial Categorization vs.Public Categorization
- Content Refresh
- Naming Conventions
- Authoring
Testing & Identifying: Issues and Inefficiencies
Usability testing can be performed in many different forms - on-site and remote. There really aren’t any limits...if you have a user, you can find a way to perform a usability test. When performing a usability test, I am always careful to explain to my user, that THEY are not the ones being tested...I am testing the site for usability, not them for proficiency. A website must be able to serve the needs of all users, regardless of a user’s computer savviness.
Testing: carrying out a usability study
- Remote (screen capture) or in person (on location, at cafe)
- Introduce yourself and explain your role
- Ask the participant to explain their role in their own words
- Ask about their general experience with the site/tool
- Ask to be walked through their daily activities
- Ask them to complete their test tasks
- Allow time for questions and feedback
Testing: flags to watch for
Some usability issues can rather obvious...but it doesn’t mean they aren’t important to note! Some common red flags to watch for...
- Clicking between open screens/tabs
- Paper ‘cheat sheets’
- Any repetitive tasks
- Unusual workarounds
- Multiple copies of content
- Using external conversion tools
- Frequently losing place or re-orienting back to home screen
Testing: review information architecture components
- Organization systems - How we categorize information
- Labelling systems - How we represent information
- Navigation systems - How we move through information
-
Searching systems - How we search information
- ie. resource search: discovering pathways example search: “vegan chocolate cake” - By ingredient: “chocolate” / By dietary consideration: “vegan” / By category: “desserts” / By keyword: “vegan cakes”
- ie. resource search: varied titles by user - Human Resources Manager, “HR Form V3051” / Team Manager, “Leave Application” / Staff Member, “going on vacation” or “holiday request”
- ie. inconsistent site-wide labels for “locations” - Branch, Store, Location, Bureaux, Airport location, Airport branch, Retail location, City Location, Boutique storefront location
Testing: Default Instructions & Help Text (opportunity for training)
- Have users contribute
- Use real language
- Hide / reveal
- New vs. power user
- People doing things differently
- Review after updates!
Documenting: How to report and recommend
With all of the hard work complete, it’s now time to organize all of your findings. I review all notes and recordings, grouping the data into categories. Using a reputable (heuristic) checklist can help expedite this process, as well as provide some evidence-based ‘oomph’ to a report. I always make sure to include screenshots and annotations in to my final reports, as many people appreciate a visual to support the recommendations.
Documenting & Reporting
- Review notes and recordings
- Group information into categories (users, features, tasks)
- Use a heuristic checklist or other best practice guide to identify/label/group issues
- Be sure to include supporting materials (eg. screenshots and transcripts)
Documenting: using a heuristic checklist (ie. Jakob Nielsen’s 10)
- Visibility of system status
- Match between system and the real world
- User control and freedom
- Consistency and standards
- Error prevention
- Recognition rather than recall
- Flexibility and efficiency of use 8. Aesthetic and minimalist design
- Help users recognize, diagnose, and recover from errors
- Help and documentation
Implementing: How to create a plan of action for improvements
Once the report is complete, it is time to meet with the development team. During this brainstorming session, I focus on what I have uncovered, presenting a prioritized (by urgency and budget) list of action items. It’s important not to overlook the input from the talented developers on your team - they may just come up with new approaches that you haven’t thought of yet.
Planning for development meeting:
- Mapped user stories to features
- Define issues by ‘what’ not ‘how’ until discussed with a dev
- Group issues by type, user, site section, and or content creation process (multiple labels ok)
- Give alternate suggestions if possible
Development planning:
- Identifying needs
- Understanding identified needs
- Isolating alternatives (prioritize by LOE and business value)
- Choosing solutions
- Refining solutions
Resources:
1. Download our free Usability Study Checklist PDF(345 KB)
2. Listen to the session recording of User Testing your Drupal Administration Process recorded live at DrupalCon Austin in June.
3. Session Slides: