A long time ago I attended a workshop with Alan Dix that applied this principle to leverage ancient (failed) wisdom for design inspiration: how to produce a good idea out of a bad idea. How does it work? You can have a look at the example Tomaso, Ting, Maria, Valentina and I worked on.

Objective:

To create a good idea starting from a bad idea; by analyzing the bad idea’s positive and negative aspects, identifying the forcings behind them, and using these forcings in a context in which they constitute positive aspects of a good idea.

The process:

We started by brainstorming for bad ideas. From these ideas we chose one, and traded the rest with other teams, receiving in turn two borrowed bad ideas. Then, we considered their positive, negative and neutral aspects, what causes the systems to behave in these ways, and how the context within the system affects our valuation (positive, neutral or negative) of these aspects.

The bad ideas:

1. Dinner payment chain: A payment system in a restaurant in which a customer pays the bill of the previous customer and cannot leave until he/she have found someone that pays his/hers.

• Pay-it-backward. As opposed to pay-it-forward, which usually requires honesty and proactivity on the part of the participant, pay-it-backward ensures that the chain will be maintained (positive). However, in this case it does not add any benefit compared to the mainstream alternative, in which each customer pays their own bill (neutral).
• Captivity within the chain. On the other hand, in this case, pay-it-backward is inconvenient for the customers who have to stay until someone pays their bill because they may have to spend time and effort looking for the next link in the chain (negative).
• Blindness about cost. Customers are also blind as to how much their meal will cost, because instead of paying the items they ordered, they will be paying another customer’s. This can cause fear to end up losing (negative).
• Snowball effect. Customers will consume a lot because they do not want to end up losing money, and for this they need to consume more than the customer for whom they are paying. In general terms, this means that as people feel they have already paid for something, they will make full use of it to feel they got their money’s worth. This is a positive for the restaurant owner that will see the business grow but a negative for the customers that will end up paying increasing amounts with each iteration.

2. Managing system for football fans (borrowed idea): A system which allows supporters to decide opponent’s team formation through voting.

• Public involvement. People participate in the decision making. They feel more engaged because the results they see are the consequence of their involvement (positive).
• People act on a narrow vision and inadvertently modify the system. People seek to satisfy immediate and individualistic goals, so they choose the worst players for the opposing team and as a result the quality of the whole league degrades (negative).

3. Beeper for restaurant kitchens (borrowed idea): A communication system for a kitchen in which all natural language communication is replaced with messages coded in “beeps”.

• More structured system. In a carefully engineered system it is easier to predict what others will do and cooperation becomes more smooth (positive), on the other hand people may feel trapped or unable to improvise (negative).
• Simplicity, limited communication and behavior. What can be said through beeps is much more limited than what can be said through natural language. This limits people’s freedom to communicate, and in turn it limits their freedom to act when they are working in a team and have to coordinate their actions (negative).
• Overly oriented to goals (avoiding socializing). This does not foster a very agreeable work environment (negative).
• Cognitive effort to remember codes. People have to learn a new language, which requires effort and raises the entry barrier (negative).
• Beeps are annoying (negative).

After the analysis, we combined some of the aspects mentioned above to develop a good idea. Below, we present the idea, and list the aspects of the bad ideas it includes. In the context of this idea, these aspects are all positive.

The good idea: A pay-it-backward vaccination campaign

Background: Vaccines do not work on all the individuals in a population. To be sure that a disease does not spread and all individuals (even those in whom the vaccine does not work) are safe, a population has to achieve herd immunity. This occurs when vaccination is widespread enough so that if one person becomes sick with an infectious disease the likelihood that he/she will be able to pass it on to another person is very low because virtually all the people around this person will be immune to the disease. This means that for an individual to be safe from infectious disease, his/her own behavior is not enough, he/she needs the collaborative behavior of the community.

The idea: We though of a system in which person-A gets vaccinated and pays for the vaccine. In turn, if person-A refers person-B to go and get the vaccine, person-A will get a refund for the cost of the vaccine. Each person wants to get a refund, what works to create a chain reaction that results in most people getting vaccinated and the population achieving herd immunity.

Below, we explain how the aspects of the bad ideas are integrated into this good idea.

• Pay-it-backward. The motivation to get a refund for something that has already been paid ensures that the chain will be maintained.
• Captivity within the chain. People have to spend time and effort looking for the next link in the chain, but this is compensated by the fact that they are sure to earn something in return (the refund).
• Blindness about cost. People pay for their vaccine and finally get a refund, so they perceive that the vaccine has a cost but that they get it for free. This has the double effect of attributing value to the vaccine and making the person feel they have drawn a benefit by not paying the cost. In reality, because all people entitled to a refund and finally the State (through a tax-funded program) ends up paying for all the vaccines, there is an unknown cost (the cost of the vaccine) paid indirectly by each person. But in this case, blindness about the real cost could be used by the people engineering the vaccination campaign to set an arbitrary “price” to the vaccine, maximizing the vaccine’s value perception and people’s motivation to get the refund.
• Public involvement. People can see in the whole of society good results that are the consequence of their involvement, this makes people feel empowered and predisposes them better to new initiatives.
• People act on a narrow vision and inadvertently modify the system. People have an immediate individual incentive (to get a refund) and the acting in pursuit of this immediate individual incentive creates a much larger collective effect: herd immunity. This occurs in this case because the individual and collective goals are aligned.
• More structured system. By creating a system in which individual and collective goals are aligned, the system can be expected to reach the desired equilibrium state without a huge external investment. The system just needs to be started and then each actor will perform its part until the whole system changes state. In this case, for example, although the State would pay for the vaccines (which we assume it would have done anyway) it would save the costs of public health campaigns, advertising, education, etc. to persuade the people to get vaccinated because each individual can be trusted to have an incentive to pass the message along for free.

I would have preferred to have a working mock up, but as we were all doing this as a side project and we had so many features, this proved impossible. So a comic had to do.  Should we have done something simpler which was easier to convey? Well, time was a constraint, there were no clear guidelines, no client, nothing at stake, so I think we just indulged: it’s a lot more fun to develop an idea than a prototype. And even if the philosophy behind the group of features we presented wasn’t explicit and we could be accused of lack of cohesion, the audience were experts in the industry, we expected them to be knowledgeable enough and trend-aware enough to put the pieces together and see the concept behind :)

You can see the prototype here.

Some months ago, I asked you to complete a questionnaire about your tabbed browsing habits. Thanks! I got 99 responses. What follows is a brief summary of how I analyzed the data, the results I got and some analysis. Later we used the results to design an alternative to tabs for the Mozilla Labs Design Challenge summer 09 (with Maria and some help from Nico), but I’ll write about that later.

First things first: where did the data come from?

• I posted a link to the questionnaire on reddit
• The link was sent to the mailing list of a software company
• Some of my colleagues also answered the questionnaire

So you can expect my data to be biased towards the savvy user’s experience, which is also something I appreciate given the nature of my research. I would like to use these results to guide me through a redesign of the current browsing paradigm (tabs). As far as I could read, there is (or there was) nothing wrong with tabs themselves; however, as people start to simultaneously open an increasingly large number of tabs and their browsing becomes more complex, tabs become simply not enough. It’s savvy users who are pushing the envelope here, and it’s them whom I mostly want to listen to this time.

How I analized the data:

To analyze the data I used Weka.  Weka is an open source software for data mining tasks, which I used in it’s GUI-like form. It contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. With it, I tried to find some patterns in my data. I don’t really believe in averaging user data, so I clustered my users/participants to find several user profiles. The idea is then to try to come up with a browsing paradigm that is flexible enough to cater, ideally, to all groups.

I used  k-means and got 4 clusters, representing 4 kinds of users (39%, 26%, 22% and 12% of participants in each respective cluster).

This is what my clusters look like:

Or, into words:

• Cluster 0: users that have mostly a limited number of tabs, 6 to 12, of which some are permanently open. Some of the organize tabs, some don’t, but overall there is no preferred organizing method for this group (methods vary across users). They mostly rely on visual cues to locate tabs, which makes sense given that the limited number of tabs enhances title legibility and increases the chance that favicons will be easily discriminable (less chance that any two or more favicons are alike).
• Cluster 1: users that have mostly more than 12 open tabs, of which some are permanently open. Some of them organize their tabs, some don’t. The ones who do seem to have a combination of methods, although they can be singled out for their preference for read/unread organization (is this because opening new tabs on the right end is a default in many browsers?). They largely rely on visual cues and trial and error to locate tabs, and (as opposed to the other group with a large number of tabs) the lack of a single/fixed/permanent semantic structure in their organization seems to hinder their ability to use associative memory for retrieval.
• Cluster 2: they use a very limited number of tabs (up to 6) and don’t have permanently opened tabs. They don’t organize their tabs and retrieve them using various methods (mostly visual cues, sequential search and localized trial and error). They don’t rely on memory, but probably they don’t need it (searching through 6 items can’t take so long).
• Cluster 3: users with more than 12 open tabs, of which some are permanently open. They all organize tabs and create a semantic structure: by subject and by same parent are very popular choices. They also organize their tabs according to frequency of use. Their use of arbitrary and associative memory is a lot higher than that of the other groups, which seems to correlate to their fixed/permanent structuring (frequency of use) and semantic structuring (by subject, by parent).  However, they also rely heavily on visual cues. Going back to the original data, everyone who listed using visual cues in this group also listed some other method, so it looks like visual has a supporting function helping the user refine the search (first letters of the title, thumbnail, favicon, etc) once memory has made the first approach.

Finally: What do I think this means?

There’s an interesting 1983 paper called “How do people organize their desks?” by Thomas Malone from the Xerox Palo Alto Research Center. This paper explores how people organize their documents (not only in their desks but also tables, shelves, cabinets, etc.) and the strategies they use to find them. Two extremes were identified. Some people just had stacks of things to do arranged in ill-defined groups, their main criteria was things that require an action of their part (and the priority of this action) vs. other things. Other people relied on information organizing and stored documents in clearly defined and titled files. This is what Malone called, respectively, piles (groups of elements arranged in no particular order) and files (groups of elements arranged with systematic order). Files are titled and things are found by looking for them in the category where they belong. Pile identification is aided by their spatial location.

Now I’m going a bit into my own interpretation of some of the things he found (like doing some unorthodox rephrasing), I think they’re definitely interesting but some of the notions need updating to fit digital platforms:

• People organize their desks so they can find things, but an equally important function of the organization is reminding. If I see it I’ll attend to it. People with messy offices do indeed have more problems finding information and remembering tasks.
• Items related to pending tasks are singled out.
• Classifying information is cognitively taxing and this difficulty is one of the forces leading the creation of loosely defined piles.
• Thinking of the context in which you last saw something helps remembering where it is. Context can be many things: who was I working with, when, what was I also working on at the time?
• Even if you organized stuff really well, that doesn’t mean that you’ll be able to find it right away: the criteria used to file it somewhere may not be the same criteria you may want to use to retrieve it (let’s say you filed according to the project the document belongs to but the want to find everything one particular person worked on). Malone found this occurred in 2/3 of the cases he analyzed. This means that organizing things doesn’t always pay, and this is why Google changed our lives at the time Yahoo was a web directory.

So, the takeaways are, basically: organizing into categories (even tagging) is hard and doesn’t always pay, users should not be forced to do it; visibility, visualization and spatial arrangeability are the killer features providing reminding power, context awareness and findability; items with pending actions should be somehow singled out; and, finally, a flexible search that does not rely on tags, labels or categories is a must.

Looking closely at clusters 1 and 3, the ones with an amount of tags that merits some organization, there’s a strange effect. The tab bar is is an unfriendly environment to organize stuff because of it’s unidimensionality, and because it does not facilitate absolute spatial arrangements at all: some wrap around creating a second/third/fourth line of tabs drastically changing a tabs position as you add more and, even worse, some extend infinitely into an imaginary space on the right outside the browser window hiding some tabs and then you navigate this macabre strip back and forth never knowing where you are. So, why am I saying this? Because it looks to me, that if technology allowed it, cluster 1 would be pilers and cluster 3 would be filers, and this, sprinkled with the joy of a flexible search engine, could be as close to browser Disneyland as it gets for tab management. Why do I think this? Well, cluster 1 people have a loose organization, based mostly on read/unread and a clearly visual in their location strategy preferences, however piles are not afforded by the current interfase so there’s no explicit grouping and no way to spatially arrange tabs so they don’t really meet the exact definiton of pilers (yet). On the other hand, cluster 2 people do organize tabs into categories in meaningful ways but this organization is not currently supported by the browser so all the labelling and category delimitation is just kept in their minds so they are not real filers (yet). And the Disneyland bit? Well, not being a cognitive psychologist or qualified in any formal way, I’m just guessing that this piler/filer stuff and the preference to spend time puting things where they belong so you know where to find them vs. to spend time searching for them is just something wired into people’s brains. I have no hard evidence, but my intuition and the fact that my results pretty much match Malone’s (in a different domain) point this way. And, wouldn’t the best interfase be the one that affords the users’ natural preferences and behavior? And, wouldn’t this be the case more so when we’re talking about preferences and behaviors that are wired into the user’s brain?

So, for the Mozilla Labs Design Challenge, our goals are:

• To build an interfase that caters to pilers and filers alike: a flexible interfase, just like Malone’s observed offices, that will afford the strategies of pilers, filers and everyone in between.
• To use the digital world’s capabilities to solve the shortcomings that these strategies have in real life.
• To use the information gathered in this study to intoduce features relevant to user preferences (permanently opened tabs, visual search, etc.)

I worked on this project, some time ago, with Wenzhu, Valentina, Paulo. When I was told we had to design a remote control for elderly people, I googled it, and then sighed. Usually I try to avoid the “yet another…” projects but this time it was not up to us, so I set to work thinking at least that socially it was quite a relevant project. Actually I learned that most developed countries have around 10% 65+ people (I knew it was a lot, but 10% is mind blowing) and that they watch TV an average of 3,5 hours a day, so it makes sense to want to adapt remote controls for their especial use.

What I didn’t know, or had never thought about, is that remote controls have been around since the 50s (commercially) and have been popular since the 70s. So these people actually lived pretty much their whole lives with remote controls, it’s different than with computers, it’s just that:

1. Remote controls have radically changed
2. Their bodies have radically changed

So we focused on this.

As a first step, we created a user profile using literature and previous studies of  remote controls for the elderly. This is it:

• Age: 60+
• Health:
  - Cognitive: Tasks that use well-practiced skills or familiar information are generally not affected by age. However, complex tasks that require taking in new information and analyzing it may become more difficult. Many researchers attribute this to deficits that occur in attention, speed of processing, and memory. Many older adults have increasing difficulty distinguishing between information that is relevant and information that is irrelevant to a particular task (attention problems). In general, memory tasks that are complex and require manipulating a lot of new information quickly become more difficult with age. Well-practiced skills and abilities remain intact.
  - Sensorial: Weak vision, more light needed to see clearly. Decreased ability to hear high frequencies and sounds in general. Low sensitivity to touch.
  - Motor: Diminished dexterity and coordination compromise fine motor skills and strength diminishes. Limited finger reach distance when gripping. Weak hand grip, slow reaction, poor accuracy.
• Gender: male/female (more female than men).
• Experience: Unfamiliar with technology in general, for most of their lifetime electronics didn’t exist or were very simple. However, they have experience of using a TV remote control. Someone who is 60, was in their 20s when the remote control was popularized, however this kind of remote controls were a lot simpler than modern remote controls.
• Goals: Change channels, adjust volume, turn on/off the TV.
• Needs:
  - Social and emotional needs: independence, a way to occupy their time / entertainment, self-confidence, inclusion.

After identifying the goals, we did a task analysis. In this case, it’s interesting because as the users have cognitive, sensorial and motor impairments the task analysis looks quite different than you’re used to. Really simple stuff can be an issue so everything has to be included, all the trivial steps being present. For example, to turn on/off the TV:

Although interpreting feedback as to whether the TV has been effectively turned on or off may not in general be a problem with the average user, it can be for elderly people. Changing channels (without zapping) is even harder:

So we did this for every goal and identified the problematic stages through an heuristic evaluation based on our users’ cognitive, sensorial and motor health. Some difficulties we identified in current remote controls are:

• The decreasing size of remote controls means small keys and small labels that people with visual disabilities find inaccessible. Some people are unable to distinguish between certain color combinations used on keypads.
• Hearing impaired users cannot identify commands or controls that require hearing, so visual or tactile feedback when keys are pressed would be recommended.
• Some current remote controls have a huge number of keys for various functions. Those with cognitive impairments may have particular difficulty in learning the function of so many keys.
• Due to reduced mobility and manual dexterity lifting and carrying a remote control or pressing small keys may prove difficult for those with physical impairments.
• The existing remote control is too large and uncomfortable for holding.
• Most elderly people often make mistakes of operation, pressing the wrong button.
• Layout of menu selection: elderly people need to memorize and navigate through the location of buttons since there are approximately 19 – 30 buttons.
• Most elderly people do not know what the outcome of their actions is, feedback is not appropriate.

And then we made some design choices. It’s strange, because if you look at our report on the design (and I’ve seen this in many reports and presentations) the fact that we made these choices and why we made them is not explicitely stated there at all. The likely reason, I think, is that many of these design choices are only supported in some obscure place in the back of our brains by anecdotal data that managed to make an impression on us. The problem is that on the way to become a UX designer we’re told a hundred times to run user tests/studies and not to trust anecdotal evidence, so we just ignore these choices, never write about them and the result is that most people don’t even realize that these were choices at all (meaning that there actually was another way to do it). However, UX design isn’t and shouldn’t be (else no innovative interfaces would ever be created) a purely scientific discipline. Let’s say that your design should always be exclusively and logically derived from user study results, well there wouldn’t be a lot of variety then. Intuition, power of observation and willingness to take some risks are as essential to UX designers as whatever knowledge about how to run a within subjects summative study and do some t-testing afterwards. Actually, IMO, anyone can read a couple of books and learn how to carry out a user test, it’s the ability to use intuition and derive successful results from it (and the ability to recall some curiosity you observed in the subway, at the right time, and use it well) that makes the good UX designer (and separates him/her from the self-proclaimed-UX-designer developers). This can only happen if you have trained you brain to think as a UX designer, then he’ll do the work for you recalling and bringing up the right idea at the right time. But you know what they say (UI developers, I’m looking at you), Rome wasn’t built in a day ;)

We didn’t have time to run a user study before going on to design the first iteration. So, a considerable number of our most important choices were based on facts like: my grandmother and Valentina’s hold the remote control in the palm on one hand and press the buttons with the index of their other hands. Just like this guy:

So we thought, wouldn’t it be nice to afford this? Most remote controls for the elderly are simplified versions of the regular ones, with bigger buttons. But the buttons are distributed all over the surface and this makes this kind of gripping very uncomfortable.

Most remote controls for the elderly have only “arrow up” and “arrow down” zapping buttons. This simplifies the interfase but makes it annoying to change from channels that are far apart. At the same time, most of the elderly we know only watch a small range of channels. So why not try some favorite buttons?

So this is what we made:

And these are its features, explained:

• Functions: favorite buttons (the green ones with varied shapes, featuring shortcuts to a set of favorite channels, based on the fact that it is likely that elderly people most usually watch a restricted set of channels repeatedly), zapping buttons, volume buttons, on/off button, sound on/off button, back button (undo one step button, that takes the user to the previous state of the system in the event of an error given that operation errors are one of our users concerns), mute button. The remote control was stripped of all other possible buttons to keep it simple and just address the primary needs of our users.
• Feedback: feedback for battery level in the back, just like the led lines in a MacBook (some elderly people have trouble changing the batteries, so it could be helpful for them to have a battery level indicator and ask relatives or caregivers to change their remote control’s batteries if they are about to be empty when they have the chance). Also, some buttons (on/off, volume, channel, back) are backlit for a small time window when pressed to indicate a change in state (this allows users to feel more in control of their actions, having better knowledge of their outcome). Some other buttons (favorite buttons and mute) stay backlit when pressed to indicate current system state.
• Shape of the remote control: Thin and narrow to be suitable to hold in the users’ palm. Anti-slipping surface for better grip. Light in weight. Wide infrared pointer that extends towards the back of the remote control (the elderly sometimes have tremors or their accuracy is diminished, so having a big pointing surface helps them to aim at the TV more easily. Also, it can be useful to people who have vision problems to be able to hold the remote towards them to better see the buttons while still pointing). A hand strap is provided to secure the remote to the hand of the user.
• Physical characteristics of the buttons: Buttons are big and well spaced to avoid operation mistakes and facilitate their recognition. They are also grouped according to functions and the layout matches usage patterns, having buttons that have to be pressed repeatedly (zapping and volume) in the most accessible positions. A concave shape for the buttons gives tactile feedback indicating the user that his/her finger is in the right position without the need of visual feedback. Haptic feedback when the buttons are pressed allows users to feel more in control of their actions, eliminating uncertainty towards the completion of an action. Buttons are soft and not much pressure is needed to press them (a decrease in hand strength is one of the signs of aging, also accuracy increases when force diminishes, so this is expected to reduce the number of operation mistakes).
• Other features: A caller is embedded in the remote, so that the user can attach a sticker button to the TV and press it to have the remote control light the buttons and beep a sound for findability (this is expected to particularly help users with memory problems to locate their remote control if they forget where they have left it).

We went to a retirement home and tested our remote control with our users (the test was controlled replicating the tasks with an ordinary remote control). The test was quite messy, because it’s not every day that some new people arrive at a retirement home and ask for help testing some piece of gadgetry ;) but our participants were super helpful and the results were quite nice :) Here they are:

1. They liked the shape of our remote control very much. They emphasized that the shape was a perfect fit for the way in which they hold the remote.
2. They thought the size of our remote control was good. Their hands were able to grab it tightly.
3. Finder function: this function was not implemented in the prototype but the idea was explained and the feedback was positive.
4. Favorite channel button. The participants’ favorite channels amount to not more than 4 in all cases so the number of favorite channel buttons implemented is enough for their requirements. The different shapes of favorite channel buttons did not confuse the users, but helped them memorize them. Feedback was positive for different shaped buttons.
5. The users did not understand the function of back button. One mentioned reason was that the text (“back”) on the button is in English. Another reason is that they confused the back button with the backwards arrow in the zapping buttons. Using “undo”, they mentioned, would be better than “back”.
6. Wider infrared pointer feature was not tested.
7. Battery level indicator: one of the users pointed out that she asked other people to change the battery for her every time. None of the users gave the feedback on whether the battery level indicator was useful for them or not.
8. Sound and light feedback on the buttons. This feature was not implemented in the remote control prototype either. The concept was explained to the users. However, it was difficult for them to understand it. In the design, once a button is pushed, this button will be backlit for 5 seconds; however, the users’ assumed that once a button is pushed, the TV screen or TV would be highlighted. Maybe with a working prototype we could be luckier :)
9. The hand strap was not implemented in our prototype. After listening to an explanation, the users thought this feature was not necessary without stating any reason.
10. The users gave the positive feedback about the buttons being big and the layout being simple. Even though there are 11 buttons only, they include all the functions the users used daily. They suggested labels would be a good addition, because they would be able to know the functions of buttons more clearly.

So, not only was the feedback positive but also our users’ preferred features were those that came up seemingly out of nowhere and that no other remote control for the elderly we surveyed had: the weird shape and the favorite buttons. And I think this is a nice morale to this story about how it’s never the case that everything has been said about something, no matter how many commercial versions are there, and not to be afraid to add some deliberate serendipity to the design process sometimes :)

Link to full report here.

In any case, last month, Toon, Dominika, Valentina, Maria and I designed and run an experiment to test if emoticons are actually the facial expressions of IM. Are they used in the same context a real facial expression would be appropriate? Is there a correlation between use of emoticons and real facial expressions produced during an IM conversation? This is how we did it and our results.

We had these two research questions:

• If the number and type of emoticons is correlated to facial expressivity (the number of instances of facial expressions) for a given instant messaging conversation.
• If emoticons aid instant messaging text communication by a mechanism equivalent to facial expressions during face to face communication (i.e. if they appear in a context when the facial expression they depict would be appropriate, natural and spontaneous).

We had the following setup. In the experiment, 10 people participate in an instant messaging conversation with an experimenter. Their faces are recorded and merged with a screen cast of the chat window, so facial expressions and the use of emoticons are recorded for each of them.

The topic of the conversations was fixed to “childhood memories”, because we thought that a topic that  involved remembrances would trigger the expression of emotions. Participants were chosen among acquaintances of the experimenters to facilitate the expression of emotions and provide a more natural setting for chatting (we read that social context is highly determinant of expressivity in computer mediated communication). Participants also had different nationalities and cultural backgrounds. And they were chosen so their ages were between 20 and 30 years old, to increase the likelihood that they are frequent users of emoticons (statistics say this age bracket is where most frequent emoticon users are).

Then we analyzed the data. We had planned to use the synchronicity of the video and the chat to see if emoticons and facial expressions were produced simultaneously. However, the software had been badly configured at the usability lab-not by us-and the videos were out of sync with the chats (we found out by chance, it was very hard to tell). On a side note, I wonder what happened with all the people who used the faulty software in the lab but never found out, what scientific breakthroughs may we be missing :P Later we found these tips for screen recording that I recommend you take a look at if you’re ever going to use Camtasia. In any case, we didn’t have synchronized videos so we had to figure out if there was anything we could do with the data we had collected.

The first research question was easy, we just had to count the number of emoticons and instances of facial expressions and see if there was a correlation. We did that, comparing the number of facial expressions with the number of emoticons sent by the participants, received by the participants (or sent to them by the experimenters) and then the total number of emoticons. Then we compared the number of emoticons in each category (smiling/joy, laughing, thinking, irony/joking, confusion, surprise, sadness, anger, disgust, fear, embarrassment) with the total number of facial expressions.  There was no correlation whatsoever: neither the number of sent/received/total emoticons nor the number of any of the different types of emoticons correlates to the facial expressivity of people during an IM conversation. Strike one.

So, fine, maybe the number or type of emoticons we send and receive has nothing to do with our general emotional expressivity during one conversation (in terms of the number of facial expressions). But it may still be that when they are used emoticons work in the same way as facial expressions (if this is not clear, you may want to pay a visit to my friend M.P. ;) This would have been very easy to prove if the videos and the chats had been synchronized (an emoticon occurrs at the same time that a facial expression or not), but because they weren’t we had to get creative.

What we did was test for a correlation between the number of emoticons in each emoticon category and the number of facial expressions that this emoticon category depicts (smile emoticons against actual smiles, etc.) and then we also counted the number of fillers and onomatopoeias in each of the conversations (you’ll see why later). If emoticons act as facial expressions, we can expect all emoticons of one type to be linked to one instance of the facial expression they depict. Also, there may be extra facial expressions of that category: the noise. You can take a look at our paper and see why we expect this noise to be randomly distributed across participants. If this noise is randomly distributed accross participants then we can still spot a trend if there is one. But again, we found no correlation at all. Although this is not as conclusive as our original experiment (the synchronous one) would have been, it seems that emoticons are not used in the same context than the facial expressions they depict. Strike two.

Finally, we found something that may not be statistically significant at all but is worth exploring in a next experiment. It looked like its possible that when people use less emoticons they use more fillers and onomatopoeias. We can’t be sure because we didn’t control the experiment for this. By visual inspection it seems that people are replacing emoticons with fillers and onomatopoeias, but when less emoticons were used the experimenters also used more fillers. The problem is that apparently people are wired to empathize through mimicking. Mimicking is an important part of making a connection during communication and we can’t help adopting some of the non-verbal quirks of our interlocutors (you can look up these people for further information on this: Buck, 1984; Burgoon et al., 1995; Chartrand & Bargh, 1999). And because we didn’t control for this variable, it’s impossible for us to say if our participants tended to use more fillers and onomatopoeias when not using so many emoticons because they are interchangeable and can thus replace one another or because they were just mimicking the experimenter :( But we think it’s worth exploring. Fillers and onomatopoeias are voicings and hence belong to the sound realm. If emoticons are interchangeable with them, it would be strike two-and-a-half.

Although the synchronous experiment is still pending, and this would be the only way to get conclusive evidence to discard that emoticons are equivalent to facial expressions, all our evidence points to emoticons and facial expressions being different things. There’s even research that suggests that people don’t even use their face-recognition machinery in the brain to decode emoticons. But, are they punctuation marks? I don’t know, some research would be needed to determine this. What would be the consequences? I don’t know either, I’m not much into affective stuff. But it looks interesting. Not only from a linguistic perspective, but also because if this is truly something new (like a new necessary disambiguation method, a new dimension to punctuation that changes the way we communicate, etc.) then undertanding how it works can inspire portability to other areas, and new features and products. Maybe the Nokia glow is just the beginning, maybe mediated communication is just the beginning of emoticons, or maybe saying “lol” out loud is just the beginning… only time will tell… how freakish our communication can get.

In the meantime, if you have some time to kill, you can download our paper form here :)

We ended up working in a kind of calendar/to-do/agenda/personal assistant application that prompts the user to carry out a task in the optimal context in time and space. This application would help the user manage his/her activities. For tasks that have a fixed scheduling, its functionality is limited to providing reminders, exactly as a regular calendar application would. However, for tasks that may be executed freely at any time before a given deadline, it suggests the best moment to do something using information about the nature of the task and the user’s preferences, live monitoring of his/her level of busyness, time, location, etc.

Before I explain how it would work, and because I can picture you shaking your head and tightening your lips, let me say something. This idea sounds familiar to you? Yes, to me too… When talking about applications that are supposed to be smart enough to suggest what you may or may not want to do, there’s a very thin line separating awesomness from complete failure. That’s what, I think, when you present an idea like this to people, makes people stand in two very distinct camps: either you love it or you hate it. There are few things, in UX, as polarizing as agents. I was very sceptical about this idea at first. Then something happened.

I realized I was being guilty of the sin of ninetiness. Was it possible that my Clippy PTSD was blurring my vision? So I tried to shake off the clippiness, and think about it in an objective way. Surely I had seen other (working) examples, but where? I know what I’m going to say may be controversial and that some people will think that I’m abusing the definition of agent. But what does actually happen every time you do a search on Google, for example? That’s no plain text search, and neither is it a text search where results are ordered according to the old version of Page Rank anymore. It’s been a long time now that Google has been taking global and local search trends, current events, your personal profile, previous search queries, search history and it appears that even websites you visit into account when choosing which results to display for your query, they even claim to be able to predict epidemic outbreaks.  According to Wikipedia (hey, I’m not an expert) there are only four essential types of intelligent software agents:

• Buyer agents or shopping bots
• User or personal agents
• Monitoring-and-surveillance agents
• Data Mining agents

If you take a look at the definition for each of them, there’s no category Google search doesn’t belong to. So it looks like this is the secret of successful agents: not looking like a paper clip, actually not looking like anything, looking like they’re not even there. If people like to have their search results tweaked if they still feel that it’s their keyword choice that brought them about, would people accept to have reminders for tasks that they never actually schedule if the system can faithfully interpret their heart’s desires? Can it be done? Well, Google does a pretty good job… so I do think it’s possible. Will it be done? I don’t know, not by me, lol. We just had to come up with a new concept that’d make our(?) lives better, and as much as I came to undertand that this is not such a bad idea, I’m not in love with the concept either, let’s just say I find it interesting.

So, how would it work? The user enters his/her schedule with fixed tasks in the same way as in a normal calendar. Also he/she enters the flexible tasks and some information about them (i.e. deadline, location, duration, sequential constraints, is this task repeating over time). The system uses location information (GPS) to learn about the user’s location, time zone and weather. Locations can be tagged (e.g. office, home, mall). Also, permanent ambient noise monitoring provides information about the current activities and level of busyness of the user (e.g. is he/she maintaining a conversation, typing on the keyboard, walking).
Finally, specific rules entered by the user and machine learning on acceptance or rejection of its suggestions provide the system with the user’s preferences. Users can also manually set states related to mood or desired level of busyness for a certain time period. All these elements are used by the system to suggest the optimal moment to carry out a task based on proximity, weather, mood, availability, etc. This is applicable to management of a daily routine, a weekly schedule and long term planning. Information related to the pending tasks (content feed) and health advice can also help the user make optimal choices about his/her workflow. Finally, casual networking and social commitments are also supported because proximity, availability and coincidence in pending tasks can be monitored.

Advantages, Challenges and Opportunities: this system has the potential to solve the problem of having to remember to look at a deadline calendar or to-do list because the user will be reminded to complete the tasks. Also, the fact that tasks are suggested in the right context will aid efficiency and avoid procrastination. The cognitive load to determine when to perform the task is switched from the user to the application. However this presents some challenges. Users will still have to manually enter information about their tasks, so the suggestions should be useful enough to justify the trade-off. Also, the application should remain non-invasive. This system also allows sharing schedules for joint tasks, displaying the user’s availability status, and the information about the user’s mood, busyness and daily tasks could potentially be used for other applications or to suggest entertainment for the user’s free time, eventually. The wide array of technical capabilities in a mobile phone provide flexible input and output modes that allow to ease task input and minimize invasiveness, respectively. The possibility to implement a multimodal interface also allows to use the system in different scenarios under different conditions.

The Great Pacific Garbage Patch is an area estimated to be twice the size of Texas that acts as purgatory for all our modern conveniences. Why take your TV to the repair shop when you can get a new one for only twice the price it’ll take to fix it? Like the new iPod 2282th generation? Non-returnable plastic bottles FTW, it’s such a hassle to take bottles back to the store…

Every day, in my job, I come across many “opportunities to make people’s lives easier”. Sometimes you get to choose what you work on, sometimes you don’t. Sometimes, like today, you have a day when you’d rather spend the largest part of it just staring through the window.

This week we have to come up with an idea, any idea, for a product that will help us improve our lives. It shouldn’t be that hard, the Professor said, “I’m sure that there are lots of things in your lives that can be made better” And I’m sure there are. My life would be better if babies didn’t get sedated for their mothers to sell fruit at the entrance of supermarkets and if iPhones and water bottles weren’t floating in a gyre in the middle of the ocean.

Natasha and I will be working during what’s left of this week in an application that allows people to scan products (using their mobile phones) and learn about the conditions in which these products were made and what is likely to happen to them at the end of their life cycle. The idea is to provide information about working conditions of the people involved in the manufacturing process, means of transportation involved in getting the product to you, overall sustainability of the production cycle, agricultural practices, if the company has some kind of recycling plan to collect devices once you want to discard them, presence of any contaminants, etc. Kind of taking Fair Trade certification to the next level, empowering people who can afford it to make a choice.

How would it work? It’ll be hard to have companies disclose that kind of information. But I’m sure you’re familiar with the nutrition facts in your cereal box, and I’m also sure McD didn’t have a ball when they had to print that on the bottom of their Big Mac box and yet it happened. Will it solve the World’s problems? Probably not. Why do I think people would care? Some, certainly, will not. This is one application that will not make your life easier: making choices and acknowledging the effect your choices have in the World is never easy. But some days, like today, I don’t need my life to be easier, I just need it to be better.

edit: I was going to delete this post, I actually did, that’s why I’m reposting. It seems after all that we will be working on something else. As I said, sometimes you get to choose, sometimes you don’t. And yes, I don’t always work on stuff that makes me proud, but that’s life :( Cat family to cheer myself up:

During our week-long course with Bert Bongers; Maria, Valentina, Dominika and I redesigned a toaster. You may also want to read part 1 and part 2 of this series of posts to learn about the concepts that we put into practice. Part 3 presents our analysis of the current toaster, and our redesign.

I’m not going to say that were doing our device parsing when we came across something interesting. I’m not going to do that because everyone knows that after you’ve toasted a first pair of slices of bread, it requires superhuman ability to get a second pair equally toasted. However, I’d like to point out that, had we though that this randomness was part of a “surprise me” feature in our toasters, we would have realized that something was wrong with the interaction anyway, when doing the device parsing.

The image on the left shows how there’s something wrong with the mapping between the input and the output, between an action on the part of the user (setting the timer) and the desired outcome (how toasted the toast is). This is a gulf of execution problem.

Depending on how hot the toaster already is and some technical characteristics, the outcome for the same knob position varies. Suposedly you can tell this because you had a look a the components, which is not really needed in the case of a toaster because I guess most people know how it works already. But, in any case, even if the output didn’t vary with variables other than the input, numbers on a knob don’t actually tell you anything about how toasted the toast will be. If you encountered this toaster for the first time, how would you know which number to choose?

So we went on to redesign the toaster for the mapping to make sense. To help the mapping, we kept the knob, but instead of having numbers we introduced the coding in the picture on the right. This is very nice as an interface, no room for ambiguity. But… how would it actually work in terms of the components of the toaster? The current toaster has a timer associated to the knob, you set it and after a while the toast is “ready” (whatever that is). Changing the numbers on the knob for pictures above will solve the mapping problem, for example for first time user of a toaster that doesn’t know what the knob is for, but will not help with the “random toastedness” factor, something inside the toaster has to change: the timer. This is the prototype for our “deterministic toaster”:

There you can see our Flash toaster (in the laptop’s screen) for which you enter how much you want to toast your bread with the knob on the bottom of the picture (to the right of the center) using the coding showed in the previous paragraph. The flashlight (blue thing on top of the plastic cup) lights the toast (green paper thing) so the webcam (white thing) can read the RGB values for the color of the toast . As bread  gets toasted, it emits different amounts of red, green blue and total light, so if you read these values you know (or the toaster knows) how toasted your bread is. We measured this in real toast, obtaining the curve on the right.

Now, replace the flashlight with well mixed RGB light, the webcam with a RGB sensor, compensate for the red glow of the resistor inside the toaster and the initial color of your bread. And when the toaster determines that the color of your toast matches your settings it can pronounce your toast “ready”.

THIS SURVEY IS NO LONGER ACTIVE

Around a month ago, I got to do my first solo project. With great power comes great responsibility and, luckily, the ability to choose the subject that better suits my interests :) And this is how I embarked on the redesign of browser error messages.

Browsers are old. It’s weird. I would have thought that major industry players would have figured out this kind of core issues by now. Error messages are not about new or experimental interactions, their design is probably as linked to basic UX design principles as can be. That’s why I was startled when I started to get the answers to my questionnaire… users mistaking DNS errors with HTTP 403 response codes and attributing SSL certificate warnings to malware. There’s a lot of educated talk on the web about how bad some error messages are, these however are savvy users’ rants and none of them come close to describing how the average users misses on their web experience, one of my participants even describing the experience as “distressing” :S

Anyway, after a week of questionnaires, qualitative analyses, guideline drafting and prototyping, I came to understand why error message design is not a popular discipline and some other interesting conclusions.

I’m posting the guidelines I came up with here, and if you feel like reading the whole analysis and taking a look at my redesign, you can download it from here.

These are my guidelines for error message design, illustrated with some examples taken from the browsers I studied (IE, Firefox, Opera and Chrome):

1. Explicitness: messages should make clear that something has happened. Either there was an error or the system is trying to warn the user because it encountered a potential threat. This error message (in the form of a Live Search result page with no results) appears in IE when the address includes a character that is not allowed as part of a URL. It fails to be explicit. The users I interviewed thought that they got this page because the browser had misinterpreted their intentions: they wanted to go to a website but the browser “thought” they were trying to look for something using the search functionality embedded in the address bar.

2. Human readability: error messages targeted to people should be readable by people. They should use human language and avoid obscure codes. This Opera error message is an example of what shouldn’t be happening: the term “illegal URL” accompanied by the ASCII code of the disallowed character carries no meaning in human colloquial speech.
3. Neutral tone and politeness: error messages should carry a neutral tone and be polite, avoiding placing the blame on the user. In this Firefox message, where it reads “did you make a mistake [...]?” could be, according to this guideline, rephrased as “are you sure this is [...]?”.

4. Precision: error messages should tell the user exactly what happened or allow them to diagnose what the problem is. As different users have different levels of expertise and facing the risk of being overly verbose, this is arguably one of the trickiest principles to master. This Chrome error fails to achieve this (even in the case when the user has typed the URL in the address bar, it mentions a broken link) while the Firefox error above was found to be too verbose for the users to read.
5. Inclusion of advice: errors should include advice for the user on what to do next, how to tackle or solve the problem. Error messages are part of a dialog that is taking place between the user and the system as a strategy leading to solving a problem that has arisen. In such a context, the responsibility of an error message designer does not end once the message is displayed. Suggesting and anticipating user response is part of error message design. When designing an error message, it is important to keep in mind that how much the user will trust an error message and will engage in performing a suggested action (presumably enhancing his/her experience) depends on the accuracy and simplicity of the advice. This Opera error prompts the user to check his/her Internet connection and this is what the users I interviewed said they would do if they got this error: “check the cables, check the router, look at the Wi-Fi icon”.  Checking if the user has an active Internet connection can be done automatically by the browser when this kind of error is triggered saving a lot of useless work and frustration on the part of the user. This guideline is probably the single most important piece of advice in the quest to create intelligent messages that can effectively dialog with users and to end the current paradigm that makes users see error messages as a dead end.
6. Focus on the user: error messages are, from the point of view of the user to whom they are targeted, a response of the system to an action performed by the user. As such, to understand from his/her point of view what caused this error and how to avoid it in the future, the user is interested in seeing a message as a response to his/her action and not an obscure technical complication that arose from the action. From the point of view of the user, the error from which Opera shows this message may either be the result of the user clicking on a broken link or the user misspelling a URL the fact that the address could not be resolved because it contained disallowed characters (although it is valuable information for the user and can help troubleshoot the problem) is not a straightforward response for his/her action.
7. Conciseness: error messages should be short and to the point. They are an interruption to the optimal process in accomplishing the user’s action and they should consume the less possible time. Lack of conciseness or overly complicated messages, can result in the user disregarding the message and abandoning their goal. In the case of this Firefox error message, for example, even if the first suggestion pointed to the right course of action, users failed to read the message at all, misidentifying the cause of the problem: they said that if they got this message they would check their Internet connection.

8. Context: error messages, when possible, should be context relevant. If the same technical complication occurred in two different scenarios (in this case, for example, when clicking on a link or typing a URL which can be discriminated by the browser), context can be the answer to providing sensible error messages. In the case of error message above, asking the user if he/she made a mistake when typing the URL is irrelevant if the user just clicked on a link, as well as in the case of this Chrome message stating that the link is broken is inaccurate if the user typed the URL.
9. Appropriateness and potentiality: error messages, and especially warnings, should convey the right degree of alarm. It should be apparent for the user that the danger involved in accepting a certificate that is valid when the date settings in his/her computer are incorrect does not offer the same security risks as accepting a certificate whose serial number was issued for a different website. All of the SLL certificate warnings of the browsers I studied (IE, Opera, Chrome and Firefox) failed to do this. This can be frustrating for the user that misses on his/her experience, or worse it can result in the user disregarding the importance of more serious warnings.
10. Clarity of origin: error messages should make clear that they are a normal part of the interaction with a system and they are not inherently something bad. As well as the user is not to blame for an error, error messages should also be designed in a way that avoids the system as being perceived as defective on the part of the user when this is not necessarily the case. For example, when users were presented this message, although it represents the intended response for the user action, users considered the system to have malfunctioned and mistaken their intentions.