Usability Test: Swimmit
Open water swim events face some surpisingly complex scheduling challenges. Like all outdoor sports, to ensure a safe, successful day of training, they must take into account the weather (temperature, chances of rain/lightning) but also wind speed, water temperature, and wave sizes, not to mention, marine life. So when I was approached by Ivan, founder of Swimmit (a meetup-style app for open water swimmers), users had been asking if the app included such information.
User Perception of the Problem
I cannot find the weather information I need.
There was just one thing...
Problem: Business Side
Users are asking for a feature that already exists.
A relatively common issue, this problem costs businesses a lot of money for their support teams to spend a dedicated chunk of time repeatedly answering each and every duplicate inquiry. The good news is that it's actually one of the better UX challenges to have because it means all the heavy (dev) lifting is complete, and the root of the problem falls somewhere under 'Presentation' - which rarely requires any particularly dev-intensive effort to resolve. From a UX standpoint, there are 4 overlapping points to consider:
Information Architecture (IA)
The weather information wasn't buried particularly deep in the app architecture, was always logically grouped with geographic locations, and occupied a prominent position in the page layout. To hypothesize, it's unlikely the IA was part of the problem, but still worth testing to ensure it matched the mental models (read: expectations) of the users.
Navigation
Here was one method to reach the weather info - though shorter variations existed:
1. Click the 'Routes' icon in the tab bar.
2. Search for a particular route.
3. Tap the search result for the individual route. The weather information is immediately visible on the loaded page.
Maintaining a relatively flat IA makes it easier to offer short, multiple navigation options, equaling a more flexible (and forgiving) environment. That said, even if it only takes a single tap or two to reach the end goal, it won't matter if users don't know what to tap, think they need to tap something else, or get confused when the underlying, step-by-step direction of logic doesn't reflect how they anticipate it should work.
User Interface Design
Here was the weather section for swim routes:
Seeing is one thing, comprehending is another. Icons are particularly dangerous without labels because unless the icon design is recognizable as an industry standard, the meaning is left entirely open to (mis)interpretation. Since some of these icons were not standardized weather icons, I worried swimmers might be having trouble understanding what they were.
Interaction Design
It's ok if your design deviates from reliable, but tired, cookie-cutter digital standards. It's not ok if your interface fails to visually prompt your users to action. Looking at the weather indicators, above, it's not visually obvious this element is interactive (i.e. it doesn't look tap-able). I wondered if, therefore, this feature was being overlooked as static content, because the default response to any interface is to do nothing.
There were actually 2 related functions:
1. If you tapped the weather indicators, the app would display the hourly forecast for that route.
2. If you tapped the circled ? (help) icon in the top right of the hourly forecast, you got a very helpful instructional page for the weather icons.
Remember my earlier concern the icons were not intuitive? This is exactly the sort of thing that could potentially resolve that issue. However, the tiny help icon size on the hourly forecast made it easy to overlook, it's not immediately obvious just by looking at the help icon what sort of help it would be providing, and ultimately, the weather icon description page was 2 taps removed from the initial weather section displayed in the swim route page.
With all that in mind, lastly, I wanted to be clear about what exactly users had been saying when asking about the weather information:
"Does this app include wave measurements?"
"Is there a way to find the wind speeds for the afternoon?"
It's hard to tell from these quotes whether users knew upfront the app contained the information they wanted, but it's clear that swimmers expected to be able to find this sort of information (or there wouldn't be so many asking about it). Simultaneously, expectations beget motivation; by taking the initiative to ask, even if users didn't know where to find it, it's safe to infer they were already looking for the information and trying to figure out where it could be. Given that this feature was already implemented, the challenge was pinpointing how, where, and when they were getting stuck.
In short, I needed to conduct a usability test. Here was my approach:
Usability Testing Tasks
1) Find the weather for the Waikiki Roughwater Swim route.
This would determine if swimmers, were, in fact, actually having trouble finding the weather information in the information architecture or getting lost in the navigation. Also, asking for a particular route made it easier to time and compare progress across each session.
2) Tell me the wind speeds for the Waikiki Roughwater Swim route for 3pm.
This task would illustrate if users inherently understood the interface elements, and if/how to interact with each one.
As this was a very short test, I also drafted some follow-up market research questions for additional strategy insights on new features and general improvement. Finally, as these sessions were entirely remote and there weren't any complicated animations or complex gesture-based interactions, I recreated the portion of the app I wanted to test using screenshots in a Marvel prototype. That way the participants could simply boot it up in their browser and share their screens so I could monitor their progress. Once 8 participants had been scheduled, I met with each for about 20min over Skype.
The Results
1) Find the weather for the Waikiki Roughwater Swim.
Everyone completed this task without issue. So users actually knew 1) where to look to find the information they were after and 2) how to get there.
2) Tell me the wind speeds for the Waikiki Roughwater Swim for 3pm.
Everyone tapped the weather icons and found the hourly weather without prompting (great!). However, when asked to identify the wind icon, there was a great deal of confusion between water temperature and wind (not so great). Additionally, not everyone knew they could tap the help icon which which would have loaded the icon description page for them.
So though the problem was initially perceived as "I cannot find the weather information I need." the actual problem was:
Problem: User Side
I don't understand what I'm looking at, and it's not obvious to me where I can go to find out.
The Root of the UX Problem
The weather icon designs are not intuitive, and swimmers don't know where to find the icon description page.
It was safe to conclude the weather section needed a bit of redesign. Luckily, there were several quick fixes that could serve as placeholders:
UX Bandaids
1) Bring the help function to the routes page. I suggested an easy way would just be to add a simple, text-based link beneath the icons that read something like, "What do these icons mean?" or "Weather Guide"
2) Add a small '>" icon to the end of the weather section to reinforce it was interactive, and help users discover the hourly forecast if they hadn't, already.
Moving Forward
While the solutions above were quick to implement, they merely patch, but do not resolve, the underlying issues. Ultimately, the icons would need either redesigned, labels, or both, and possibly changed from a single-line element into, say, an accordion style element or some other navigation layout that hides or reveals additional, optional information. Real estate is at a premium on mobile devices, but people are willing to endure additional taps and scrolling, provided they feel they are progressing in their pursuits [1]. To address the root of the problem, a more elegant redesign would need to cleanly balance the two.
1. "Interaction Elasticity", JAKOB NIELSEN, https://www.nngroup.com/articles/interaction-elasticity/
Usability Test: Swimmit
Open water swim events face some surpisingly complex scheduling challenges. Like all outdoor sports, to ensure a safe, successful day of training, they must take into account the weather (temperature, chances of rain/lightning) but also wind speed, water temperature, and wave sizes, not to mention, marine life. So when I was approached by Ivan, founder of Swimmit (a meetup-style app for open water swimmers), users had been asking if the app included such information.
User Perception of the Problem
I cannot find the weather information I need.
There was just one thing...
Problem: Business Side
Users are asking for a feature that already exists.
A relatively common issue, this problem costs businesses a lot of money for their support teams to spend a dedicated chunk of time repeatedly answering each and every duplicate inquiry. The good news is that it's actually one of the better UX challenges to have because it means all the heavy (dev) lifting is complete, and the root of the problem falls somewhere under 'Presentation' - which rarely requires any particularly dev-intensive effort to resolve. From a UX standpoint, there are 4 overlapping points to consider:
Information Architecture (IA)
The weather information wasn't buried particularly deep in the app architecture, was always logically grouped with geographic locations, and occupied a prominent position in the page layout. To hypothesize, it's unlikely the IA was part of the problem, but still worth testing to ensure it matched the mental models (read: expectations) of the users.
Navigation
Here was one method to reach the weather info - though shorter variations existed:
1. Click the 'Routes' icon in the tab bar.
2. Search for a particular route.
3. Tap the search result for the individual route. The weather information is immediately visible on the loaded page.
Maintaining a relatively flat IA makes it easier to offer short, multiple navigation options, equaling a more flexible (and forgiving) environment. That said, even if it only takes a single tap or two to reach the end goal, it won't matter if users don't know what to tap, think they need to tap something else, or get confused when the underlying, step-by-step direction of logic doesn't reflect how they anticipate it should work.
User Interface Design
Here was the weather section for swim routes:
Seeing is one thing, comprehending is another. Icons are particularly dangerous without labels because unless the icon design is recognizable as an industry standard, the meaning is left entirely open to (mis)interpretation. Since some of these icons were not standardized weather icons, I worried swimmers might be having trouble understanding what they were.
Interaction Design
It's ok if your design deviates from reliable, but tired, cookie-cutter digital standards. It's not ok if your interface fails to visually prompt your users to action. Looking at the weather indicators, above, it's not visually obvious this element is interactive (i.e. it doesn't look tap-able). I wondered if, therefore, this feature was being overlooked as static content, because the default response to any interface is to do nothing.
There were actually 2 related functions:
1. If you tapped the weather indicators, the app would display the hourly forecast for that route.
2. If you tapped the circled ? (help) icon in the top right of the hourly forecast, you got a very helpful instructional page for the weather icons.
Remember my earlier concern the icons were not intuitive? This is exactly the sort of thing that could potentially resolve that issue. However, the tiny help icon size on the hourly forecast made it easy to overlook, it's not immediately obvious just by looking at the help icon what sort of help it would be providing, and ultimately, the weather icon description page was 2 taps removed from the initial weather section displayed in the swim route page.
With all that in mind, lastly, I wanted to be clear about what exactly users had been saying when asking about the weather information:
"Does this app include wave measurements?"
"Is there a way to find the wind speeds for the afternoon?"
It's hard to tell from these quotes whether users knew upfront the app contained the information they wanted, but it's clear that swimmers expected to be able to find this sort of information (or there wouldn't be so many asking about it). Simultaneously, expectations beget motivation; by taking the initiative to ask, even if users didn't know where to find it, it's safe to infer they were already looking for the information and trying to figure out where it could be. Given that this feature was already implemented, the challenge was pinpointing how, where, and when they were getting stuck.
In short, I needed to conduct a usability test. Here was my approach:
Usability Testing Tasks
1) Find the weather for the Waikiki Roughwater Swim route.
This would determine if swimmers, were, in fact, actually having trouble finding the weather information in the information architecture or getting lost in the navigation. Also, asking for a particular route made it easier to time and compare progress across each session.
2) Tell me the wind speeds for the Waikiki Roughwater Swim route for 3pm.
This task would illustrate if users inherently understood the interface elements, and if/how to interact with each one.
As this was a very short test, I also drafted some follow-up market research questions for additional strategy insights on new features and general improvement. Finally, as these sessions were entirely remote and there weren't any complicated animations or complex gesture-based interactions, I recreated the portion of the app I wanted to test using screenshots in a Marvel prototype. That way the participants could simply boot it up in their browser and share their screens so I could monitor their progress. Once 8 participants had been scheduled, I met with each for about 20min over Skype.
The Results
1) Find the weather for the Waikiki Roughwater Swim.
Everyone completed this task without issue. So users actually knew 1) where to look to find the information they were after and 2) how to get there.
2) Tell me the wind speeds for the Waikiki Roughwater Swim for 3pm.
Everyone tapped the weather icons and found the hourly weather without prompting (great!). However, when asked to identify the wind icon, there was a great deal of confusion between water temperature and wind (not so great). Additionally, not everyone knew they could tap the help icon which which would have loaded the icon description page for them.
So though the problem was initially perceived as "I cannot find the weather information I need." the actual problem was:
Problem: User Side
I don't understand what I'm looking at, and it's not obvious to me where I can go to find out.
The Root of the UX Problem
The weather icon designs are not intuitive, and swimmers don't know where to find the icon description page.
It was safe to conclude the weather section needed a bit of redesign. Luckily, there were several quick fixes that could serve as placeholders:
UX Bandaids
1) Bring the help function to the routes page. I suggested an easy way would just be to add a simple, text-based link beneath the icons that read something like, "What do these icons mean?" or "Weather Guide"
2) Add a small '>" icon to the end of the weather section to reinforce it was interactive, and help users discover the hourly forecast if they hadn't, already.
Moving Forward
While the solutions above were quick to implement, they merely patch, but do not resolve, the underlying issues. Ultimately, the icons would need either redesigned, labels, or both, and possibly changed from a single-line element into, say, an accordion style element or some other navigation layout that hides or reveals additional, optional information. Real estate is at a premium on mobile devices, but people are willing to endure additional taps and scrolling, provided they feel they are progressing in their pursuits [1]. To address the root of the problem, a more elegant redesign would need to cleanly balance the two.
1. "Interaction Elasticity", JAKOB NIELSEN, https://www.nngroup.com/articles/interaction-elasticity/