1. A thorough gathering of requirements should include the following questions: What should the system do? But also… What could it do? What must it do? What must it not do? Nice piece of advice borrowed from my dark (hard-core engineering) past, who would have thought it’d turn out to be so useful in design.
2. Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes (Goodhart’s law). What does this mean? That relationships between two variables in historical data (as for example between speed and user satisfaction) will not necessarily hold after you start acting on one of them to influence the other one (like in: increasing speed past a certain limit will no longer produce an effect in user satisfaction).
3. Conway’s law ([...] organizations which design systems [...] are constrained to produce designs which are copies of the communication structures of these organizations) is not meant to be humorous! Evidence courtesy of Microsoft Research.
4. Good design needs not be perfect. At one time, Westinghouse began to work on heat pumps that could provide heating and cooling, and believed that he might be able to extract enough power in the process for the system to run itself. Lord Kelvin told him that he would be violating the laws of thermodynamics. Westinghouse replied that might be the case, but it made no difference. If he couldn’t build a perpetual-motion machine, he would still have a heat pump system that he could patent and sell. The rest is history.

A long time ago I attended a workshop with Alan Dix that applied this principle to leverage ancient (failed) wisdom for design inspiration: how to produce a good idea out of a bad idea. How does it work? You can have a look at the example Tomaso, Ting, Maria, Valentina and I worked on.

Objective:

To create a good idea starting from a bad idea; by analyzing the bad idea’s positive and negative aspects, identifying the forcings behind them, and using these forcings in a context in which they constitute positive aspects of a good idea.

The process:

We started by brainstorming for bad ideas. From these ideas we chose one, and traded the rest with other teams, receiving in turn two borrowed bad ideas. Then, we considered their positive, negative and neutral aspects, what causes the systems to behave in these ways, and how the context within the system affects our valuation (positive, neutral or negative) of these aspects.

The bad ideas:

1. Dinner payment chain: A payment system in a restaurant in which a customer pays the bill of the previous customer and cannot leave until he/she have found someone that pays his/hers.

• Pay-it-backward. As opposed to pay-it-forward, which usually requires honesty and proactivity on the part of the participant, pay-it-backward ensures that the chain will be maintained (positive). However, in this case it does not add any benefit compared to the mainstream alternative, in which each customer pays their own bill (neutral).
• Captivity within the chain. On the other hand, in this case, pay-it-backward is inconvenient for the customers who have to stay until someone pays their bill because they may have to spend time and effort looking for the next link in the chain (negative).
• Blindness about cost. Customers are also blind as to how much their meal will cost, because instead of paying the items they ordered, they will be paying another customer’s. This can cause fear to end up losing (negative).
• Snowball effect. Customers will consume a lot because they do not want to end up losing money, and for this they need to consume more than the customer for whom they are paying. In general terms, this means that as people feel they have already paid for something, they will make full use of it to feel they got their money’s worth. This is a positive for the restaurant owner that will see the business grow but a negative for the customers that will end up paying increasing amounts with each iteration.

2. Managing system for football fans (borrowed idea): A system which allows supporters to decide opponent’s team formation through voting.

• Public involvement. People participate in the decision making. They feel more engaged because the results they see are the consequence of their involvement (positive).
• People act on a narrow vision and inadvertently modify the system. People seek to satisfy immediate and individualistic goals, so they choose the worst players for the opposing team and as a result the quality of the whole league degrades (negative).

3. Beeper for restaurant kitchens (borrowed idea): A communication system for a kitchen in which all natural language communication is replaced with messages coded in “beeps”.

• More structured system. In a carefully engineered system it is easier to predict what others will do and cooperation becomes more smooth (positive), on the other hand people may feel trapped or unable to improvise (negative).
• Simplicity, limited communication and behavior. What can be said through beeps is much more limited than what can be said through natural language. This limits people’s freedom to communicate, and in turn it limits their freedom to act when they are working in a team and have to coordinate their actions (negative).
• Overly oriented to goals (avoiding socializing). This does not foster a very agreeable work environment (negative).
• Cognitive effort to remember codes. People have to learn a new language, which requires effort and raises the entry barrier (negative).
• Beeps are annoying (negative).

After the analysis, we combined some of the aspects mentioned above to develop a good idea. Below, we present the idea, and list the aspects of the bad ideas it includes. In the context of this idea, these aspects are all positive.

The good idea: A pay-it-backward vaccination campaign

Background: Vaccines do not work on all the individuals in a population. To be sure that a disease does not spread and all individuals (even those in whom the vaccine does not work) are safe, a population has to achieve herd immunity. This occurs when vaccination is widespread enough so that if one person becomes sick with an infectious disease the likelihood that he/she will be able to pass it on to another person is very low because virtually all the people around this person will be immune to the disease. This means that for an individual to be safe from infectious disease, his/her own behavior is not enough, he/she needs the collaborative behavior of the community.

The idea: We though of a system in which person-A gets vaccinated and pays for the vaccine. In turn, if person-A refers person-B to go and get the vaccine, person-A will get a refund for the cost of the vaccine. Each person wants to get a refund, what works to create a chain reaction that results in most people getting vaccinated and the population achieving herd immunity.

Below, we explain how the aspects of the bad ideas are integrated into this good idea.

• Pay-it-backward. The motivation to get a refund for something that has already been paid ensures that the chain will be maintained.
• Captivity within the chain. People have to spend time and effort looking for the next link in the chain, but this is compensated by the fact that they are sure to earn something in return (the refund).
• Blindness about cost. People pay for their vaccine and finally get a refund, so they perceive that the vaccine has a cost but that they get it for free. This has the double effect of attributing value to the vaccine and making the person feel they have drawn a benefit by not paying the cost. In reality, because all people entitled to a refund and finally the State (through a tax-funded program) ends up paying for all the vaccines, there is an unknown cost (the cost of the vaccine) paid indirectly by each person. But in this case, blindness about the real cost could be used by the people engineering the vaccination campaign to set an arbitrary “price” to the vaccine, maximizing the vaccine’s value perception and people’s motivation to get the refund.
• Public involvement. People can see in the whole of society good results that are the consequence of their involvement, this makes people feel empowered and predisposes them better to new initiatives.
• People act on a narrow vision and inadvertently modify the system. People have an immediate individual incentive (to get a refund) and the acting in pursuit of this immediate individual incentive creates a much larger collective effect: herd immunity. This occurs in this case because the individual and collective goals are aligned.
• More structured system. By creating a system in which individual and collective goals are aligned, the system can be expected to reach the desired equilibrium state without a huge external investment. The system just needs to be started and then each actor will perform its part until the whole system changes state. In this case, for example, although the State would pay for the vaccines (which we assume it would have done anyway) it would save the costs of public health campaigns, advertising, education, etc. to persuade the people to get vaccinated because each individual can be trusted to have an incentive to pass the message along for free.

I want to write a full post about prototype fidelity and testing methods later, and some (approximate) guidelines about when to do which thing… so I’ll try to keep digression to a minimum ;) Now, without further ado… my humble best attempt at explaining why qualitative research can be objective, reliable and produce useful insights about how users experience systems and products (and take that look off your face already, I can see it in my crystal ball and I’m not liking it ;)

First I’ll go with my favorite argument for qualitative research: all that glitters is not gold. So, if your product is on the early stages of development, it is reasonable to expect that new insights on how users interact with it will result in substantial changes. Leaving aside the 0% of the cases in which you got it right from the start (ask Mr. Gall), you *want* to be in for substantial changes in the beginning, you want to experiment and get it wrong, learn form it and bake this wisdom into your successive iterations. They key here is the word “substantial” and my assumption that you don’t have 3.7 billion years to throw away in evolution Darwinian-style. What I mean, is that you could always arbitrarily branch your project, do some quantitative research to find what works better, stick to that branch and from there repeat endlessly… 100 million species stand as evidence that this method works. But wouldn’t it be nice to know *why* and *how* something is better than some other thing and do a bit of selective breeding? (at this point I would expect your disbelief to have turned into sporadic nods) The thing is that, quantitative research, as many hard numbers as it can provide, and statistical significance and alphas and betas, can’t say anything about why something happens. It’s power is limited to who did what, where and when (ok, I’ll admit to *procedural* how too, after all it’s just a sequence of whats). Why someone did something or how he/she reached the conclusion that this is what had to be done is out of the scope of quantitative research. This doesn’t mean that we can’t get the hard numbers and then make some educated guesses about why and how things happened. But guess what? We would always be guessing ;) Qualitative analysis, which has become the black sheep of methods for actually coming out on the fact that we do interpret what we see, actually does more observing to back up this why and how interpretation than quantitative research does.

So, how do we cope with the fact that yes, we are interpreting and we’re smearing our preconceptions, our desires, our imperfections onto the facts from which we want to draw objective conclusions? Most of the criticism on qualitative research comes from bad bad stuff that happened in the 70s, when some social scientists presumably tired of abusing hard drugs moved into abusing research methods. But that doesn’t mean that now we don’t have standards to ensure that whatever influence a scientist has, it’s counterbalanced by other scientists, weeded out by the use of common classification methods (coding schemes, ontologies) or at least reflected upon, acknowledged and noted as possible weakness of the research process. Of course no one is perfect, but this also applies to quantitative research as well, and today there are countless journals and conferences that accept qualitative research papers.

Below, there are some of the methods scientists can use to safeguard the quality of their qualitative results:

• During the data collection process, log data on video/audio/etc to make sure that it’s accessible to multiple scientists for later analysis.
• If you’re going to use one, decide on a coding scheme to classify your observations and stick to it.
• Have multiple scientists collect data and if possible have multiple scientists poll the same test sources to correct any systematic bias.
• To analyze the data, always recruit more than one scientist and try to get people with differing positions.
• If in doubt, find some knowledgeable outsider to oversee your process.
• Always support your conclusions with excerpts from your original data.
• Never cite any numerical results. There are no “4 people out of 5″ in qualitative analysis. Things like “most of the people we observed” are OK because they can serve as honest leads for future direction, I trust you’ll know the difference.
• Be honest :)

I’m working on a project that involves some ergonomics, more specifically it requires or would benefit from the label “comfortable”. Like we always do, we designed a test, collected participants, drafted consent forms, prepared the facilities… and then… the unexpected. To my embarrassment, we had to repeat our whole biomechanics experiment because we had gathered our results in a manner that didn’t afford any meaningful analysis. This is the brave account of what went wrong and how we solved it, which I send into the world hoping that at least one less designer will stumble against this cheeky stone ;)

Like I said, our goal was to determine if a particular physical interaction we were designing was “comfortable”. So, what did we do? I’m not going to explain exactly in what the experiment consisted, but the idea was have people try it and then use some validated questionnaires to tell us if they had felt physical discomfort during or after the tasks we proposed. The questionnaire had a scale with a few ordinal values (uncomfortable, moderately uncomfortable, slightly uncomfortable, comfortable). What can go wrong? Well, in the first place, we were surprised to see that our participants had go through considerable more discomfort than expected during our experiment. We were confused because we had tried the experiment while designing it and none of us had had *any* discomfort whatsoever. Still it could be that overall the system was comfortable enough, we could still do some kind of analysis: we had our ordinal variables… Here was where our lucky misfortune saved us from dataitis (dataitis is what you get when you forget that information is data+meaning, and I can’t emphasize that “+meaning” enough). The crazy discomfort outcome made us uneasy, clearly something had happened there, maybe Hawthorne but also maybe something else. We started questioning our method and the someone said

and what if it actually is uncomfortable? what if all [interactions of this kind] are uncomfortable? what if for [this kind of interaction] comfortable just means less uncomfortable than average?

This made sense, maybe when we tried the experiment we hadn’t found it uncomfortable because we knew in which context the interaction belonged and our users didn’t. But this also opened a whole new set of questions:

• What is comfortable? How should we measure it?
• If all interactions of this kind are uncomfortable, and we measure with our ordinate categorical scale and aggregate the results, we’re going to find that our interaction is indeed uncomfortable to some degree but does it mean that it’s not comfortable enough?
• And even worse, if all interactions of this kind (including ours) are comfortable, and we determine that our interaction is indeed comfortable through our experiment, will it still be comfortable out there in the market is someone does it better?

We were lucky, if the results had been positive, we would have never reflected on this: if you’re going to measure a subjective quality you have to do it in comparison to something else. In other words: if there’s no scale, you have to create your own. Users’ pronouncements on subjective qualities can measure the improvement of a product over time because there’s a past to which new results can be compared, and they can only measure how good a product is if there are other products to compare with.

So what we did was: we repeated the same test with two additional alternatives for the kind of interaction that we wanted to test. We did a within-user randomized test (with 12 participants), and asked users to rank all three interactions in the comfort scale. But there’s another tricky bit yet to come… how do we analyze the data? In these cases, one can be tempted to do the following things (all examples of things I’ve seen done, and even published!)

• Convert the ordinal variables to numerical scores and use a Wilcoxon signed-rank test. This would be wrong because… the fact that you express an ordinal scale in a way that looks more like an interval scale does NOT turn you data into interval variables!! A scale that goes from uncomfortable to comfortable is not, and will never be, an interval scale because the *difference* between a value and the one immediately following is undefined. The only thing we know is that for each individual, “slightly uncomfortable” means more comfortable than “moderately uncomfortable” and this is it, we don’t know and there is no way to know how much more.
• A t-test would not only be wrong on the same grounds as the Wilcoxon signed-rank, but also because you can’t assume the distribution to be normal. Using dependent t-tests in cases like this is something I’ve seen done and published many times :-(
• The Mann-Whitney U test. Mann-Whitney is at least a non-parametric test. This means it works for ordinal data. However Mann-Whitney requires *mutual independence within and between samples*, which is not the case here. As the results where gathered in a within-user test, the way participants used the scale depended on their appreciation of the range of comfort provided by the three interactions and the rank they gave each interaction was definitely affected by its comparison to the other two. So Mann-Whitney is not a choice.

Maybe there are many right choices (and maybe you can think of more possible wrong choices), but this is what we did: a Friedman test. The Friedman test is a non-parametric test used to compare observations repeated on the same subjects. The Friedman test is probably the littlest-known piece of math by Nobel prize winner economist Milton Friedman, you just have to have a look at the newspapers to see why people cared more about his demonstration of the complexity of stabilization policy… but for User Experience the Friedman test is key. A test that can answer a simple but powerful question:

N users rate k different products. Are any products ranked consistently higher or lower than the others?

I would have preferred to have a working mock up, but as we were all doing this as a side project and we had so many features, this proved impossible. So a comic had to do.  Should we have done something simpler which was easier to convey? Well, time was a constraint, there were no clear guidelines, no client, nothing at stake, so I think we just indulged: it’s a lot more fun to develop an idea than a prototype. And even if the philosophy behind the group of features we presented wasn’t explicit and we could be accused of lack of cohesion, the audience were experts in the industry, we expected them to be knowledgeable enough and trend-aware enough to put the pieces together and see the concept behind :)

You can see the prototype here.

Some months ago, I asked you to complete a questionnaire about your tabbed browsing habits. Thanks! I got 99 responses. What follows is a brief summary of how I analyzed the data, the results I got and some analysis. Later we used the results to design an alternative to tabs for the Mozilla Labs Design Challenge summer 09 (with Maria and some help from Nico), but I’ll write about that later.

First things first: where did the data come from?

• I posted a link to the questionnaire on reddit
• The link was sent to the mailing list of a software company
• Some of my colleagues also answered the questionnaire

So you can expect my data to be biased towards the savvy user’s experience, which is also something I appreciate given the nature of my research. I would like to use these results to guide me through a redesign of the current browsing paradigm (tabs). As far as I could read, there is (or there was) nothing wrong with tabs themselves; however, as people start to simultaneously open an increasingly large number of tabs and their browsing becomes more complex, tabs become simply not enough. It’s savvy users who are pushing the envelope here, and it’s them whom I mostly want to listen to this time.

How I analized the data:

To analyze the data I used Weka.  Weka is an open source software for data mining tasks, which I used in it’s GUI-like form. It contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. With it, I tried to find some patterns in my data. I don’t really believe in averaging user data, so I clustered my users/participants to find several user profiles. The idea is then to try to come up with a browsing paradigm that is flexible enough to cater, ideally, to all groups.

I used  k-means and got 4 clusters, representing 4 kinds of users (39%, 26%, 22% and 12% of participants in each respective cluster).

This is what my clusters look like:

Or, into words:

• Cluster 0: users that have mostly a limited number of tabs, 6 to 12, of which some are permanently open. Some of the organize tabs, some don’t, but overall there is no preferred organizing method for this group (methods vary across users). They mostly rely on visual cues to locate tabs, which makes sense given that the limited number of tabs enhances title legibility and increases the chance that favicons will be easily discriminable (less chance that any two or more favicons are alike).
• Cluster 1: users that have mostly more than 12 open tabs, of which some are permanently open. Some of them organize their tabs, some don’t. The ones who do seem to have a combination of methods, although they can be singled out for their preference for read/unread organization (is this because opening new tabs on the right end is a default in many browsers?). They largely rely on visual cues and trial and error to locate tabs, and (as opposed to the other group with a large number of tabs) the lack of a single/fixed/permanent semantic structure in their organization seems to hinder their ability to use associative memory for retrieval.
• Cluster 2: they use a very limited number of tabs (up to 6) and don’t have permanently opened tabs. They don’t organize their tabs and retrieve them using various methods (mostly visual cues, sequential search and localized trial and error). They don’t rely on memory, but probably they don’t need it (searching through 6 items can’t take so long).
• Cluster 3: users with more than 12 open tabs, of which some are permanently open. They all organize tabs and create a semantic structure: by subject and by same parent are very popular choices. They also organize their tabs according to frequency of use. Their use of arbitrary and associative memory is a lot higher than that of the other groups, which seems to correlate to their fixed/permanent structuring (frequency of use) and semantic structuring (by subject, by parent).  However, they also rely heavily on visual cues. Going back to the original data, everyone who listed using visual cues in this group also listed some other method, so it looks like visual has a supporting function helping the user refine the search (first letters of the title, thumbnail, favicon, etc) once memory has made the first approach.

Finally: What do I think this means?

There’s an interesting 1983 paper called “How do people organize their desks?” by Thomas Malone from the Xerox Palo Alto Research Center. This paper explores how people organize their documents (not only in their desks but also tables, shelves, cabinets, etc.) and the strategies they use to find them. Two extremes were identified. Some people just had stacks of things to do arranged in ill-defined groups, their main criteria was things that require an action of their part (and the priority of this action) vs. other things. Other people relied on information organizing and stored documents in clearly defined and titled files. This is what Malone called, respectively, piles (groups of elements arranged in no particular order) and files (groups of elements arranged with systematic order). Files are titled and things are found by looking for them in the category where they belong. Pile identification is aided by their spatial location.

Now I’m going a bit into my own interpretation of some of the things he found (like doing some unorthodox rephrasing), I think they’re definitely interesting but some of the notions need updating to fit digital platforms:

• People organize their desks so they can find things, but an equally important function of the organization is reminding. If I see it I’ll attend to it. People with messy offices do indeed have more problems finding information and remembering tasks.
• Items related to pending tasks are singled out.
• Classifying information is cognitively taxing and this difficulty is one of the forces leading the creation of loosely defined piles.
• Thinking of the context in which you last saw something helps remembering where it is. Context can be many things: who was I working with, when, what was I also working on at the time?
• Even if you organized stuff really well, that doesn’t mean that you’ll be able to find it right away: the criteria used to file it somewhere may not be the same criteria you may want to use to retrieve it (let’s say you filed according to the project the document belongs to but the want to find everything one particular person worked on). Malone found this occurred in 2/3 of the cases he analyzed. This means that organizing things doesn’t always pay, and this is why Google changed our lives at the time Yahoo was a web directory.

So, the takeaways are, basically: organizing into categories (even tagging) is hard and doesn’t always pay, users should not be forced to do it; visibility, visualization and spatial arrangeability are the killer features providing reminding power, context awareness and findability; items with pending actions should be somehow singled out; and, finally, a flexible search that does not rely on tags, labels or categories is a must.

Looking closely at clusters 1 and 3, the ones with an amount of tags that merits some organization, there’s a strange effect. The tab bar is is an unfriendly environment to organize stuff because of it’s unidimensionality, and because it does not facilitate absolute spatial arrangements at all: some wrap around creating a second/third/fourth line of tabs drastically changing a tabs position as you add more and, even worse, some extend infinitely into an imaginary space on the right outside the browser window hiding some tabs and then you navigate this macabre strip back and forth never knowing where you are. So, why am I saying this? Because it looks to me, that if technology allowed it, cluster 1 would be pilers and cluster 3 would be filers, and this, sprinkled with the joy of a flexible search engine, could be as close to browser Disneyland as it gets for tab management. Why do I think this? Well, cluster 1 people have a loose organization, based mostly on read/unread and a clearly visual in their location strategy preferences, however piles are not afforded by the current interfase so there’s no explicit grouping and no way to spatially arrange tabs so they don’t really meet the exact definiton of pilers (yet). On the other hand, cluster 2 people do organize tabs into categories in meaningful ways but this organization is not currently supported by the browser so all the labelling and category delimitation is just kept in their minds so they are not real filers (yet). And the Disneyland bit? Well, not being a cognitive psychologist or qualified in any formal way, I’m just guessing that this piler/filer stuff and the preference to spend time puting things where they belong so you know where to find them vs. to spend time searching for them is just something wired into people’s brains. I have no hard evidence, but my intuition and the fact that my results pretty much match Malone’s (in a different domain) point this way. And, wouldn’t the best interfase be the one that affords the users’ natural preferences and behavior? And, wouldn’t this be the case more so when we’re talking about preferences and behaviors that are wired into the user’s brain?

So, for the Mozilla Labs Design Challenge, our goals are:

• To build an interfase that caters to pilers and filers alike: a flexible interfase, just like Malone’s observed offices, that will afford the strategies of pilers, filers and everyone in between.
• To use the digital world’s capabilities to solve the shortcomings that these strategies have in real life.
• To use the information gathered in this study to intoduce features relevant to user preferences (permanently opened tabs, visual search, etc.)

I worked on this project, some time ago, with Wenzhu, Valentina, Paulo. When I was told we had to design a remote control for elderly people, I googled it, and then sighed. Usually I try to avoid the “yet another…” projects but this time it was not up to us, so I set to work thinking at least that socially it was quite a relevant project. Actually I learned that most developed countries have around 10% 65+ people (I knew it was a lot, but 10% is mind blowing) and that they watch TV an average of 3,5 hours a day, so it makes sense to want to adapt remote controls for their especial use.

What I didn’t know, or had never thought about, is that remote controls have been around since the 50s (commercially) and have been popular since the 70s. So these people actually lived pretty much their whole lives with remote controls, it’s different than with computers, it’s just that:

1. Remote controls have radically changed
2. Their bodies have radically changed

So we focused on this.

As a first step, we created a user profile using literature and previous studies of  remote controls for the elderly. This is it:

• Age: 60+
• Health:
  - Cognitive: Tasks that use well-practiced skills or familiar information are generally not affected by age. However, complex tasks that require taking in new information and analyzing it may become more difficult. Many researchers attribute this to deficits that occur in attention, speed of processing, and memory. Many older adults have increasing difficulty distinguishing between information that is relevant and information that is irrelevant to a particular task (attention problems). In general, memory tasks that are complex and require manipulating a lot of new information quickly become more difficult with age. Well-practiced skills and abilities remain intact.
  - Sensorial: Weak vision, more light needed to see clearly. Decreased ability to hear high frequencies and sounds in general. Low sensitivity to touch.
  - Motor: Diminished dexterity and coordination compromise fine motor skills and strength diminishes. Limited finger reach distance when gripping. Weak hand grip, slow reaction, poor accuracy.
• Gender: male/female (more female than men).
• Experience: Unfamiliar with technology in general, for most of their lifetime electronics didn’t exist or were very simple. However, they have experience of using a TV remote control. Someone who is 60, was in their 20s when the remote control was popularized, however this kind of remote controls were a lot simpler than modern remote controls.
• Goals: Change channels, adjust volume, turn on/off the TV.
• Needs:
  - Social and emotional needs: independence, a way to occupy their time / entertainment, self-confidence, inclusion.

After identifying the goals, we did a task analysis. In this case, it’s interesting because as the users have cognitive, sensorial and motor impairments the task analysis looks quite different than you’re used to. Really simple stuff can be an issue so everything has to be included, all the trivial steps being present. For example, to turn on/off the TV:

Although interpreting feedback as to whether the TV has been effectively turned on or off may not in general be a problem with the average user, it can be for elderly people. Changing channels (without zapping) is even harder:

So we did this for every goal and identified the problematic stages through an heuristic evaluation based on our users’ cognitive, sensorial and motor health. Some difficulties we identified in current remote controls are:

• The decreasing size of remote controls means small keys and small labels that people with visual disabilities find inaccessible. Some people are unable to distinguish between certain color combinations used on keypads.
• Hearing impaired users cannot identify commands or controls that require hearing, so visual or tactile feedback when keys are pressed would be recommended.
• Some current remote controls have a huge number of keys for various functions. Those with cognitive impairments may have particular difficulty in learning the function of so many keys.
• Due to reduced mobility and manual dexterity lifting and carrying a remote control or pressing small keys may prove difficult for those with physical impairments.
• The existing remote control is too large and uncomfortable for holding.
• Most elderly people often make mistakes of operation, pressing the wrong button.
• Layout of menu selection: elderly people need to memorize and navigate through the location of buttons since there are approximately 19 – 30 buttons.
• Most elderly people do not know what the outcome of their actions is, feedback is not appropriate.

And then we made some design choices. It’s strange, because if you look at our report on the design (and I’ve seen this in many reports and presentations) the fact that we made these choices and why we made them is not explicitely stated there at all. The likely reason, I think, is that many of these design choices are only supported in some obscure place in the back of our brains by anecdotal data that managed to make an impression on us. The problem is that on the way to become a UX designer we’re told a hundred times to run user tests/studies and not to trust anecdotal evidence, so we just ignore these choices, never write about them and the result is that most people don’t even realize that these were choices at all (meaning that there actually was another way to do it). However, UX design isn’t and shouldn’t be (else no innovative interfaces would ever be created) a purely scientific discipline. Let’s say that your design should always be exclusively and logically derived from user study results, well there wouldn’t be a lot of variety then. Intuition, power of observation and willingness to take some risks are as essential to UX designers as whatever knowledge about how to run a within subjects summative study and do some t-testing afterwards. Actually, IMO, anyone can read a couple of books and learn how to carry out a user test, it’s the ability to use intuition and derive successful results from it (and the ability to recall some curiosity you observed in the subway, at the right time, and use it well) that makes the good UX designer (and separates him/her from the self-proclaimed-UX-designer developers). This can only happen if you have trained you brain to think as a UX designer, then he’ll do the work for you recalling and bringing up the right idea at the right time. But you know what they say (UI developers, I’m looking at you), Rome wasn’t built in a day ;)

We didn’t have time to run a user study before going on to design the first iteration. So, a considerable number of our most important choices were based on facts like: my grandmother and Valentina’s hold the remote control in the palm on one hand and press the buttons with the index of their other hands. Just like this guy:

So we thought, wouldn’t it be nice to afford this? Most remote controls for the elderly are simplified versions of the regular ones, with bigger buttons. But the buttons are distributed all over the surface and this makes this kind of gripping very uncomfortable.

Most remote controls for the elderly have only “arrow up” and “arrow down” zapping buttons. This simplifies the interfase but makes it annoying to change from channels that are far apart. At the same time, most of the elderly we know only watch a small range of channels. So why not try some favorite buttons?

So this is what we made:

And these are its features, explained:

• Functions: favorite buttons (the green ones with varied shapes, featuring shortcuts to a set of favorite channels, based on the fact that it is likely that elderly people most usually watch a restricted set of channels repeatedly), zapping buttons, volume buttons, on/off button, sound on/off button, back button (undo one step button, that takes the user to the previous state of the system in the event of an error given that operation errors are one of our users concerns), mute button. The remote control was stripped of all other possible buttons to keep it simple and just address the primary needs of our users.
• Feedback: feedback for battery level in the back, just like the led lines in a MacBook (some elderly people have trouble changing the batteries, so it could be helpful for them to have a battery level indicator and ask relatives or caregivers to change their remote control’s batteries if they are about to be empty when they have the chance). Also, some buttons (on/off, volume, channel, back) are backlit for a small time window when pressed to indicate a change in state (this allows users to feel more in control of their actions, having better knowledge of their outcome). Some other buttons (favorite buttons and mute) stay backlit when pressed to indicate current system state.
• Shape of the remote control: Thin and narrow to be suitable to hold in the users’ palm. Anti-slipping surface for better grip. Light in weight. Wide infrared pointer that extends towards the back of the remote control (the elderly sometimes have tremors or their accuracy is diminished, so having a big pointing surface helps them to aim at the TV more easily. Also, it can be useful to people who have vision problems to be able to hold the remote towards them to better see the buttons while still pointing). A hand strap is provided to secure the remote to the hand of the user.
• Physical characteristics of the buttons: Buttons are big and well spaced to avoid operation mistakes and facilitate their recognition. They are also grouped according to functions and the layout matches usage patterns, having buttons that have to be pressed repeatedly (zapping and volume) in the most accessible positions. A concave shape for the buttons gives tactile feedback indicating the user that his/her finger is in the right position without the need of visual feedback. Haptic feedback when the buttons are pressed allows users to feel more in control of their actions, eliminating uncertainty towards the completion of an action. Buttons are soft and not much pressure is needed to press them (a decrease in hand strength is one of the signs of aging, also accuracy increases when force diminishes, so this is expected to reduce the number of operation mistakes).
• Other features: A caller is embedded in the remote, so that the user can attach a sticker button to the TV and press it to have the remote control light the buttons and beep a sound for findability (this is expected to particularly help users with memory problems to locate their remote control if they forget where they have left it).

We went to a retirement home and tested our remote control with our users (the test was controlled replicating the tasks with an ordinary remote control). The test was quite messy, because it’s not every day that some new people arrive at a retirement home and ask for help testing some piece of gadgetry ;) but our participants were super helpful and the results were quite nice :) Here they are:

1. They liked the shape of our remote control very much. They emphasized that the shape was a perfect fit for the way in which they hold the remote.
2. They thought the size of our remote control was good. Their hands were able to grab it tightly.
3. Finder function: this function was not implemented in the prototype but the idea was explained and the feedback was positive.
4. Favorite channel button. The participants’ favorite channels amount to not more than 4 in all cases so the number of favorite channel buttons implemented is enough for their requirements. The different shapes of favorite channel buttons did not confuse the users, but helped them memorize them. Feedback was positive for different shaped buttons.
5. The users did not understand the function of back button. One mentioned reason was that the text (“back”) on the button is in English. Another reason is that they confused the back button with the backwards arrow in the zapping buttons. Using “undo”, they mentioned, would be better than “back”.
6. Wider infrared pointer feature was not tested.
7. Battery level indicator: one of the users pointed out that she asked other people to change the battery for her every time. None of the users gave the feedback on whether the battery level indicator was useful for them or not.
8. Sound and light feedback on the buttons. This feature was not implemented in the remote control prototype either. The concept was explained to the users. However, it was difficult for them to understand it. In the design, once a button is pushed, this button will be backlit for 5 seconds; however, the users’ assumed that once a button is pushed, the TV screen or TV would be highlighted. Maybe with a working prototype we could be luckier :)
9. The hand strap was not implemented in our prototype. After listening to an explanation, the users thought this feature was not necessary without stating any reason.
10. The users gave the positive feedback about the buttons being big and the layout being simple. Even though there are 11 buttons only, they include all the functions the users used daily. They suggested labels would be a good addition, because they would be able to know the functions of buttons more clearly.

So, not only was the feedback positive but also our users’ preferred features were those that came up seemingly out of nowhere and that no other remote control for the elderly we surveyed had: the weird shape and the favorite buttons. And I think this is a nice morale to this story about how it’s never the case that everything has been said about something, no matter how many commercial versions are there, and not to be afraid to add some deliberate serendipity to the design process sometimes :)

Link to full report here.

In any case, last month, Toon, Dominika, Valentina, Maria and I designed and run an experiment to test if emoticons are actually the facial expressions of IM. Are they used in the same context a real facial expression would be appropriate? Is there a correlation between use of emoticons and real facial expressions produced during an IM conversation? This is how we did it and our results.

We had these two research questions:

• If the number and type of emoticons is correlated to facial expressivity (the number of instances of facial expressions) for a given instant messaging conversation.
• If emoticons aid instant messaging text communication by a mechanism equivalent to facial expressions during face to face communication (i.e. if they appear in a context when the facial expression they depict would be appropriate, natural and spontaneous).

We had the following setup. In the experiment, 10 people participate in an instant messaging conversation with an experimenter. Their faces are recorded and merged with a screen cast of the chat window, so facial expressions and the use of emoticons are recorded for each of them.

The topic of the conversations was fixed to “childhood memories”, because we thought that a topic that  involved remembrances would trigger the expression of emotions. Participants were chosen among acquaintances of the experimenters to facilitate the expression of emotions and provide a more natural setting for chatting (we read that social context is highly determinant of expressivity in computer mediated communication). Participants also had different nationalities and cultural backgrounds. And they were chosen so their ages were between 20 and 30 years old, to increase the likelihood that they are frequent users of emoticons (statistics say this age bracket is where most frequent emoticon users are).

Then we analyzed the data. We had planned to use the synchronicity of the video and the chat to see if emoticons and facial expressions were produced simultaneously. However, the software had been badly configured at the usability lab-not by us-and the videos were out of sync with the chats (we found out by chance, it was very hard to tell). On a side note, I wonder what happened with all the people who used the faulty software in the lab but never found out, what scientific breakthroughs may we be missing :P Later we found these tips for screen recording that I recommend you take a look at if you’re ever going to use Camtasia. In any case, we didn’t have synchronized videos so we had to figure out if there was anything we could do with the data we had collected.

The first research question was easy, we just had to count the number of emoticons and instances of facial expressions and see if there was a correlation. We did that, comparing the number of facial expressions with the number of emoticons sent by the participants, received by the participants (or sent to them by the experimenters) and then the total number of emoticons. Then we compared the number of emoticons in each category (smiling/joy, laughing, thinking, irony/joking, confusion, surprise, sadness, anger, disgust, fear, embarrassment) with the total number of facial expressions.  There was no correlation whatsoever: neither the number of sent/received/total emoticons nor the number of any of the different types of emoticons correlates to the facial expressivity of people during an IM conversation. Strike one.

So, fine, maybe the number or type of emoticons we send and receive has nothing to do with our general emotional expressivity during one conversation (in terms of the number of facial expressions). But it may still be that when they are used emoticons work in the same way as facial expressions (if this is not clear, you may want to pay a visit to my friend M.P. ;) This would have been very easy to prove if the videos and the chats had been synchronized (an emoticon occurrs at the same time that a facial expression or not), but because they weren’t we had to get creative.

What we did was test for a correlation between the number of emoticons in each emoticon category and the number of facial expressions that this emoticon category depicts (smile emoticons against actual smiles, etc.) and then we also counted the number of fillers and onomatopoeias in each of the conversations (you’ll see why later). If emoticons act as facial expressions, we can expect all emoticons of one type to be linked to one instance of the facial expression they depict. Also, there may be extra facial expressions of that category: the noise. You can take a look at our paper and see why we expect this noise to be randomly distributed across participants. If this noise is randomly distributed accross participants then we can still spot a trend if there is one. But again, we found no correlation at all. Although this is not as conclusive as our original experiment (the synchronous one) would have been, it seems that emoticons are not used in the same context than the facial expressions they depict. Strike two.

Finally, we found something that may not be statistically significant at all but is worth exploring in a next experiment. It looked like its possible that when people use less emoticons they use more fillers and onomatopoeias. We can’t be sure because we didn’t control the experiment for this. By visual inspection it seems that people are replacing emoticons with fillers and onomatopoeias, but when less emoticons were used the experimenters also used more fillers. The problem is that apparently people are wired to empathize through mimicking. Mimicking is an important part of making a connection during communication and we can’t help adopting some of the non-verbal quirks of our interlocutors (you can look up these people for further information on this: Buck, 1984; Burgoon et al., 1995; Chartrand & Bargh, 1999). And because we didn’t control for this variable, it’s impossible for us to say if our participants tended to use more fillers and onomatopoeias when not using so many emoticons because they are interchangeable and can thus replace one another or because they were just mimicking the experimenter :( But we think it’s worth exploring. Fillers and onomatopoeias are voicings and hence belong to the sound realm. If emoticons are interchangeable with them, it would be strike two-and-a-half.

Although the synchronous experiment is still pending, and this would be the only way to get conclusive evidence to discard that emoticons are equivalent to facial expressions, all our evidence points to emoticons and facial expressions being different things. There’s even research that suggests that people don’t even use their face-recognition machinery in the brain to decode emoticons. But, are they punctuation marks? I don’t know, some research would be needed to determine this. What would be the consequences? I don’t know either, I’m not much into affective stuff. But it looks interesting. Not only from a linguistic perspective, but also because if this is truly something new (like a new necessary disambiguation method, a new dimension to punctuation that changes the way we communicate, etc.) then undertanding how it works can inspire portability to other areas, and new features and products. Maybe the Nokia glow is just the beginning, maybe mediated communication is just the beginning of emoticons, or maybe saying “lol” out loud is just the beginning… only time will tell… how freakish our communication can get.

In the meantime, if you have some time to kill, you can download our paper form here :)

We ended up working in a kind of calendar/to-do/agenda/personal assistant application that prompts the user to carry out a task in the optimal context in time and space. This application would help the user manage his/her activities. For tasks that have a fixed scheduling, its functionality is limited to providing reminders, exactly as a regular calendar application would. However, for tasks that may be executed freely at any time before a given deadline, it suggests the best moment to do something using information about the nature of the task and the user’s preferences, live monitoring of his/her level of busyness, time, location, etc.

Before I explain how it would work, and because I can picture you shaking your head and tightening your lips, let me say something. This idea sounds familiar to you? Yes, to me too… When talking about applications that are supposed to be smart enough to suggest what you may or may not want to do, there’s a very thin line separating awesomness from complete failure. That’s what, I think, when you present an idea like this to people, makes people stand in two very distinct camps: either you love it or you hate it. There are few things, in UX, as polarizing as agents. I was very sceptical about this idea at first. Then something happened.

I realized I was being guilty of the sin of ninetiness. Was it possible that my Clippy PTSD was blurring my vision? So I tried to shake off the clippiness, and think about it in an objective way. Surely I had seen other (working) examples, but where? I know what I’m going to say may be controversial and that some people will think that I’m abusing the definition of agent. But what does actually happen every time you do a search on Google, for example? That’s no plain text search, and neither is it a text search where results are ordered according to the old version of Page Rank anymore. It’s been a long time now that Google has been taking global and local search trends, current events, your personal profile, previous search queries, search history and it appears that even websites you visit into account when choosing which results to display for your query, they even claim to be able to predict epidemic outbreaks.  According to Wikipedia (hey, I’m not an expert) there are only four essential types of intelligent software agents:

• Buyer agents or shopping bots
• User or personal agents
• Monitoring-and-surveillance agents
• Data Mining agents

If you take a look at the definition for each of them, there’s no category Google search doesn’t belong to. So it looks like this is the secret of successful agents: not looking like a paper clip, actually not looking like anything, looking like they’re not even there. If people like to have their search results tweaked if they still feel that it’s their keyword choice that brought them about, would people accept to have reminders for tasks that they never actually schedule if the system can faithfully interpret their heart’s desires? Can it be done? Well, Google does a pretty good job… so I do think it’s possible. Will it be done? I don’t know, not by me, lol. We just had to come up with a new concept that’d make our(?) lives better, and as much as I came to undertand that this is not such a bad idea, I’m not in love with the concept either, let’s just say I find it interesting.

So, how would it work? The user enters his/her schedule with fixed tasks in the same way as in a normal calendar. Also he/she enters the flexible tasks and some information about them (i.e. deadline, location, duration, sequential constraints, is this task repeating over time). The system uses location information (GPS) to learn about the user’s location, time zone and weather. Locations can be tagged (e.g. office, home, mall). Also, permanent ambient noise monitoring provides information about the current activities and level of busyness of the user (e.g. is he/she maintaining a conversation, typing on the keyboard, walking).
Finally, specific rules entered by the user and machine learning on acceptance or rejection of its suggestions provide the system with the user’s preferences. Users can also manually set states related to mood or desired level of busyness for a certain time period. All these elements are used by the system to suggest the optimal moment to carry out a task based on proximity, weather, mood, availability, etc. This is applicable to management of a daily routine, a weekly schedule and long term planning. Information related to the pending tasks (content feed) and health advice can also help the user make optimal choices about his/her workflow. Finally, casual networking and social commitments are also supported because proximity, availability and coincidence in pending tasks can be monitored.

Advantages, Challenges and Opportunities: this system has the potential to solve the problem of having to remember to look at a deadline calendar or to-do list because the user will be reminded to complete the tasks. Also, the fact that tasks are suggested in the right context will aid efficiency and avoid procrastination. The cognitive load to determine when to perform the task is switched from the user to the application. However this presents some challenges. Users will still have to manually enter information about their tasks, so the suggestions should be useful enough to justify the trade-off. Also, the application should remain non-invasive. This system also allows sharing schedules for joint tasks, displaying the user’s availability status, and the information about the user’s mood, busyness and daily tasks could potentially be used for other applications or to suggest entertainment for the user’s free time, eventually. The wide array of technical capabilities in a mobile phone provide flexible input and output modes that allow to ease task input and minimize invasiveness, respectively. The possibility to implement a multimodal interface also allows to use the system in different scenarios under different conditions.

The Great Pacific Garbage Patch is an area estimated to be twice the size of Texas that acts as purgatory for all our modern conveniences. Why take your TV to the repair shop when you can get a new one for only twice the price it’ll take to fix it? Like the new iPod 2282th generation? Non-returnable plastic bottles FTW, it’s such a hassle to take bottles back to the store…

Every day, in my job, I come across many “opportunities to make people’s lives easier”. Sometimes you get to choose what you work on, sometimes you don’t. Sometimes, like today, you have a day when you’d rather spend the largest part of it just staring through the window.

This week we have to come up with an idea, any idea, for a product that will help us improve our lives. It shouldn’t be that hard, the Professor said, “I’m sure that there are lots of things in your lives that can be made better” And I’m sure there are. My life would be better if babies didn’t get sedated for their mothers to sell fruit at the entrance of supermarkets and if iPhones and water bottles weren’t floating in a gyre in the middle of the ocean.

Natasha and I will be working during what’s left of this week in an application that allows people to scan products (using their mobile phones) and learn about the conditions in which these products were made and what is likely to happen to them at the end of their life cycle. The idea is to provide information about working conditions of the people involved in the manufacturing process, means of transportation involved in getting the product to you, overall sustainability of the production cycle, agricultural practices, if the company has some kind of recycling plan to collect devices once you want to discard them, presence of any contaminants, etc. Kind of taking Fair Trade certification to the next level, empowering people who can afford it to make a choice.

How would it work? It’ll be hard to have companies disclose that kind of information. But I’m sure you’re familiar with the nutrition facts in your cereal box, and I’m also sure McD didn’t have a ball when they had to print that on the bottom of their Big Mac box and yet it happened. Will it solve the World’s problems? Probably not. Why do I think people would care? Some, certainly, will not. This is one application that will not make your life easier: making choices and acknowledging the effect your choices have in the World is never easy. But some days, like today, I don’t need my life to be easier, I just need it to be better.

edit: I was going to delete this post, I actually did, that’s why I’m reposting. It seems after all that we will be working on something else. As I said, sometimes you get to choose, sometimes you don’t. And yes, I don’t always work on stuff that makes me proud, but that’s life :( Cat family to cheer myself up: