I would have preferred to have a working mock up, but as we were all doing this as a side project and we had so many features, this proved impossible. So a comic had to do.  Should we have done something simpler which was easier to convey? Well, time was a constraint, there were no clear guidelines, no client, nothing at stake, so I think we just indulged: it’s a lot more fun to develop an idea than a prototype. And even if the philosophy behind the group of features we presented wasn’t explicit and we could be accused of lack of cohesion, the audience were experts in the industry, we expected them to be knowledgeable enough and trend-aware enough to put the pieces together and see the concept behind :)

You can see the prototype here.

Some months ago, I asked you to complete a questionnaire about your tabbed browsing habits. Thanks! I got 99 responses. What follows is a brief summary of how I analyzed the data, the results I got and some analysis. Later we used the results to design an alternative to tabs for the Mozilla Labs Design Challenge summer 09 (with Maria and some help from Nico), but I’ll write about that later.

First things first: where did the data come from?

• I posted a link to the questionnaire on reddit
• The link was sent to the mailing list of a software company
• Some of my colleagues also answered the questionnaire

So you can expect my data to be biased towards the savvy user’s experience, which is also something I appreciate given the nature of my research. I would like to use these results to guide me through a redesign of the current browsing paradigm (tabs). As far as I could read, there is (or there was) nothing wrong with tabs themselves; however, as people start to simultaneously open an increasingly large number of tabs and their browsing becomes more complex, tabs become simply not enough. It’s savvy users who are pushing the envelope here, and it’s them whom I mostly want to listen to this time.

How I analized the data:

To analyze the data I used Weka.  Weka is an open source software for data mining tasks, which I used in it’s GUI-like form. It contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. With it, I tried to find some patterns in my data. I don’t really believe in averaging user data, so I clustered my users/participants to find several user profiles. The idea is then to try to come up with a browsing paradigm that is flexible enough to cater, ideally, to all groups.

I used  k-means and got 4 clusters, representing 4 kinds of users (39%, 26%, 22% and 12% of participants in each respective cluster).

This is what my clusters look like:

Or, into words:

• Cluster 0: users that have mostly a limited number of tabs, 6 to 12, of which some are permanently open. Some of the organize tabs, some don’t, but overall there is no preferred organizing method for this group (methods vary across users). They mostly rely on visual cues to locate tabs, which makes sense given that the limited number of tabs enhances title legibility and increases the chance that favicons will be easily discriminable (less chance that any two or more favicons are alike).
• Cluster 1: users that have mostly more than 12 open tabs, of which some are permanently open. Some of them organize their tabs, some don’t. The ones who do seem to have a combination of methods, although they can be singled out for their preference for read/unread organization (is this because opening new tabs on the right end is a default in many browsers?). They largely rely on visual cues and trial and error to locate tabs, and (as opposed to the other group with a large number of tabs) the lack of a single/fixed/permanent semantic structure in their organization seems to hinder their ability to use associative memory for retrieval.
• Cluster 2: they use a very limited number of tabs (up to 6) and don’t have permanently opened tabs. They don’t organize their tabs and retrieve them using various methods (mostly visual cues, sequential search and localized trial and error). They don’t rely on memory, but probably they don’t need it (searching through 6 items can’t take so long).
• Cluster 3: users with more than 12 open tabs, of which some are permanently open. They all organize tabs and create a semantic structure: by subject and by same parent are very popular choices. They also organize their tabs according to frequency of use. Their use of arbitrary and associative memory is a lot higher than that of the other groups, which seems to correlate to their fixed/permanent structuring (frequency of use) and semantic structuring (by subject, by parent).  However, they also rely heavily on visual cues. Going back to the original data, everyone who listed using visual cues in this group also listed some other method, so it looks like visual has a supporting function helping the user refine the search (first letters of the title, thumbnail, favicon, etc) once memory has made the first approach.

Finally: What do I think this means?

There’s an interesting 1983 paper called “How do people organize their desks?” by Thomas Malone from the Xerox Palo Alto Research Center. This paper explores how people organize their documents (not only in their desks but also tables, shelves, cabinets, etc.) and the strategies they use to find them. Two extremes were identified. Some people just had stacks of things to do arranged in ill-defined groups, their main criteria was things that require an action of their part (and the priority of this action) vs. other things. Other people relied on information organizing and stored documents in clearly defined and titled files. This is what Malone called, respectively, piles (groups of elements arranged in no particular order) and files (groups of elements arranged with systematic order). Files are titled and things are found by looking for them in the category where they belong. Pile identification is aided by their spatial location.

Now I’m going a bit into my own interpretation of some of the things he found (like doing some unorthodox rephrasing), I think they’re definitely interesting but some of the notions need updating to fit digital platforms:

• People organize their desks so they can find things, but an equally important function of the organization is reminding. If I see it I’ll attend to it. People with messy offices do indeed have more problems finding information and remembering tasks.
• Items related to pending tasks are singled out.
• Classifying information is cognitively taxing and this difficulty is one of the forces leading the creation of loosely defined piles.
• Thinking of the context in which you last saw something helps remembering where it is. Context can be many things: who was I working with, when, what was I also working on at the time?
• Even if you organized stuff really well, that doesn’t mean that you’ll be able to find it right away: the criteria used to file it somewhere may not be the same criteria you may want to use to retrieve it (let’s say you filed according to the project the document belongs to but the want to find everything one particular person worked on). Malone found this occurred in 2/3 of the cases he analyzed. This means that organizing things doesn’t always pay, and this is why Google changed our lives at the time Yahoo was a web directory.

So, the takeaways are, basically: organizing into categories (even tagging) is hard and doesn’t always pay, users should not be forced to do it; visibility, visualization and spatial arrangeability are the killer features providing reminding power, context awareness and findability; items with pending actions should be somehow singled out; and, finally, a flexible search that does not rely on tags, labels or categories is a must.

Looking closely at clusters 1 and 3, the ones with an amount of tags that merits some organization, there’s a strange effect. The tab bar is is an unfriendly environment to organize stuff because of it’s unidimensionality, and because it does not facilitate absolute spatial arrangements at all: some wrap around creating a second/third/fourth line of tabs drastically changing a tabs position as you add more and, even worse, some extend infinitely into an imaginary space on the right outside the browser window hiding some tabs and then you navigate this macabre strip back and forth never knowing where you are. So, why am I saying this? Because it looks to me, that if technology allowed it, cluster 1 would be pilers and cluster 3 would be filers, and this, sprinkled with the joy of a flexible search engine, could be as close to browser Disneyland as it gets for tab management. Why do I think this? Well, cluster 1 people have a loose organization, based mostly on read/unread and a clearly visual in their location strategy preferences, however piles are not afforded by the current interfase so there’s no explicit grouping and no way to spatially arrange tabs so they don’t really meet the exact definiton of pilers (yet). On the other hand, cluster 2 people do organize tabs into categories in meaningful ways but this organization is not currently supported by the browser so all the labelling and category delimitation is just kept in their minds so they are not real filers (yet). And the Disneyland bit? Well, not being a cognitive psychologist or qualified in any formal way, I’m just guessing that this piler/filer stuff and the preference to spend time puting things where they belong so you know where to find them vs. to spend time searching for them is just something wired into people’s brains. I have no hard evidence, but my intuition and the fact that my results pretty much match Malone’s (in a different domain) point this way. And, wouldn’t the best interfase be the one that affords the users’ natural preferences and behavior? And, wouldn’t this be the case more so when we’re talking about preferences and behaviors that are wired into the user’s brain?

So, for the Mozilla Labs Design Challenge, our goals are:

• To build an interfase that caters to pilers and filers alike: a flexible interfase, just like Malone’s observed offices, that will afford the strategies of pilers, filers and everyone in between.
• To use the digital world’s capabilities to solve the shortcomings that these strategies have in real life.
• To use the information gathered in this study to intoduce features relevant to user preferences (permanently opened tabs, visual search, etc.)

THIS SURVEY IS NO LONGER ACTIVE

Around a month ago, I got to do my first solo project. With great power comes great responsibility and, luckily, the ability to choose the subject that better suits my interests :) And this is how I embarked on the redesign of browser error messages.

Browsers are old. It’s weird. I would have thought that major industry players would have figured out this kind of core issues by now. Error messages are not about new or experimental interactions, their design is probably as linked to basic UX design principles as can be. That’s why I was startled when I started to get the answers to my questionnaire… users mistaking DNS errors with HTTP 403 response codes and attributing SSL certificate warnings to malware. There’s a lot of educated talk on the web about how bad some error messages are, these however are savvy users’ rants and none of them come close to describing how the average users misses on their web experience, one of my participants even describing the experience as “distressing” :S

Anyway, after a week of questionnaires, qualitative analyses, guideline drafting and prototyping, I came to understand why error message design is not a popular discipline and some other interesting conclusions.

I’m posting the guidelines I came up with here, and if you feel like reading the whole analysis and taking a look at my redesign, you can download it from here.

These are my guidelines for error message design, illustrated with some examples taken from the browsers I studied (IE, Firefox, Opera and Chrome):

1. Explicitness: messages should make clear that something has happened. Either there was an error or the system is trying to warn the user because it encountered a potential threat. This error message (in the form of a Live Search result page with no results) appears in IE when the address includes a character that is not allowed as part of a URL. It fails to be explicit. The users I interviewed thought that they got this page because the browser had misinterpreted their intentions: they wanted to go to a website but the browser “thought” they were trying to look for something using the search functionality embedded in the address bar.

2. Human readability: error messages targeted to people should be readable by people. They should use human language and avoid obscure codes. This Opera error message is an example of what shouldn’t be happening: the term “illegal URL” accompanied by the ASCII code of the disallowed character carries no meaning in human colloquial speech.
3. Neutral tone and politeness: error messages should carry a neutral tone and be polite, avoiding placing the blame on the user. In this Firefox message, where it reads “did you make a mistake [...]?” could be, according to this guideline, rephrased as “are you sure this is [...]?”.

4. Precision: error messages should tell the user exactly what happened or allow them to diagnose what the problem is. As different users have different levels of expertise and facing the risk of being overly verbose, this is arguably one of the trickiest principles to master. This Chrome error fails to achieve this (even in the case when the user has typed the URL in the address bar, it mentions a broken link) while the Firefox error above was found to be too verbose for the users to read.
5. Inclusion of advice: errors should include advice for the user on what to do next, how to tackle or solve the problem. Error messages are part of a dialog that is taking place between the user and the system as a strategy leading to solving a problem that has arisen. In such a context, the responsibility of an error message designer does not end once the message is displayed. Suggesting and anticipating user response is part of error message design. When designing an error message, it is important to keep in mind that how much the user will trust an error message and will engage in performing a suggested action (presumably enhancing his/her experience) depends on the accuracy and simplicity of the advice. This Opera error prompts the user to check his/her Internet connection and this is what the users I interviewed said they would do if they got this error: “check the cables, check the router, look at the Wi-Fi icon”.  Checking if the user has an active Internet connection can be done automatically by the browser when this kind of error is triggered saving a lot of useless work and frustration on the part of the user. This guideline is probably the single most important piece of advice in the quest to create intelligent messages that can effectively dialog with users and to end the current paradigm that makes users see error messages as a dead end.
6. Focus on the user: error messages are, from the point of view of the user to whom they are targeted, a response of the system to an action performed by the user. As such, to understand from his/her point of view what caused this error and how to avoid it in the future, the user is interested in seeing a message as a response to his/her action and not an obscure technical complication that arose from the action. From the point of view of the user, the error from which Opera shows this message may either be the result of the user clicking on a broken link or the user misspelling a URL the fact that the address could not be resolved because it contained disallowed characters (although it is valuable information for the user and can help troubleshoot the problem) is not a straightforward response for his/her action.
7. Conciseness: error messages should be short and to the point. They are an interruption to the optimal process in accomplishing the user’s action and they should consume the less possible time. Lack of conciseness or overly complicated messages, can result in the user disregarding the message and abandoning their goal. In the case of this Firefox error message, for example, even if the first suggestion pointed to the right course of action, users failed to read the message at all, misidentifying the cause of the problem: they said that if they got this message they would check their Internet connection.

8. Context: error messages, when possible, should be context relevant. If the same technical complication occurred in two different scenarios (in this case, for example, when clicking on a link or typing a URL which can be discriminated by the browser), context can be the answer to providing sensible error messages. In the case of error message above, asking the user if he/she made a mistake when typing the URL is irrelevant if the user just clicked on a link, as well as in the case of this Chrome message stating that the link is broken is inaccurate if the user typed the URL.
9. Appropriateness and potentiality: error messages, and especially warnings, should convey the right degree of alarm. It should be apparent for the user that the danger involved in accepting a certificate that is valid when the date settings in his/her computer are incorrect does not offer the same security risks as accepting a certificate whose serial number was issued for a different website. All of the SLL certificate warnings of the browsers I studied (IE, Opera, Chrome and Firefox) failed to do this. This can be frustrating for the user that misses on his/her experience, or worse it can result in the user disregarding the importance of more serious warnings.
10. Clarity of origin: error messages should make clear that they are a normal part of the interaction with a system and they are not inherently something bad. As well as the user is not to blame for an error, error messages should also be designed in a way that avoids the system as being perceived as defective on the part of the user when this is not necessarily the case. For example, when users were presented this message, although it represents the intended response for the user action, users considered the system to have malfunctioned and mistaken their intentions.