20 replies to this topic

[cre8pc]

This was too good to pass up...

Source: DUX 2007: A great conference, but fundamentally off the mark by Bob Jacobson of Total Experience.

He's at the conference and reports his impressions on the experience of a conference about user experience.

“A user's perception of a device or system” seems a peculiarly narrow niche in which to ply one's experience design skills. Of course, it's important: devices and systems are what drive the machinery of commerce and government, and even how we as consumers conduct ourselves at home and in leisure time. But so mechanistic a conception of the human being is antithetical to our knowledge of how people holistically perceive, think, act, and experience their lives. Maybe that's why Don himself on more than one public occasion has lamented his invention of the term, “user experience design,” suggesting we'd be better off without the “user.”

and

Perhaps it's a function of the organizing process, but it appears to me that with only a few exceptions, most of the speakers and workshop leaders -- and I suppose, attendees -- appear to be shy of 40 years of age. That means they would have been born sometime after 1967, when systemic thinking was king and every person was treated as a cog in some larger device; and that they came of age in the mid-80s or later, as information technology was replacing systems as the predominant archetypal metaphor.

My own observations and personal feelings are that sooner or later end users will stop basing their experiences on the short-lived thrill of the next rollout of the "something new". There's a movement towards substance and the "integrity of being" as I call it. The impact of the "green" movement tells me that people are ready for experiences that place a strong value and emphasis on their participation and programs that include and welcome them, rather than being a "cog in the wheel".

Designing for participation can be seen in social media, but despite all these new sites designed to bring us together, I still feel disconnected. The experience of social networking is only going to be based on how much we're willing to share. Rather than the whole human, we're more likely to get bits and pieces and believe we're getting a human experience online. We're not.

Consider that playing out right now is the fight over what comes "first" - SEO or usability. The whole argument leaps right over the idea of creating something meaningful. Holly Buchanan wrote about the CVS marketing campaign that, for her, portrays an experience that in no way resembles a real human experience.

Are we not designing or even marketing for the human experience anymore and if so, why not?

[SEOigloo]

Gosh, what an interesting topic, Kim!

It may be self-defeating to say this, but I believe I've found some of the best information on some of the most poorly designed websites which were created out of novice efforts with almost no apparent thought for user experience. It's taught me just how willing to dig a person can be if they want information badly enough. I believe I relate to the Internet, largely, the way that humans have traditionally related to books.

Reading one of the classic works of literature, science, spirituality, etc. is the chance to go one on one with a great mind. I seek out information on the Internet to go one on one with the creator of it in that same way, and though a scary website does put off the SEO in me, it is the contents of what is offered that I am attempting to experience.

That being said, it's obviously going to be a better experience if I can find what I'm looking for.

I bring this same attitude to the very well designed blogs/forums/SM sites. Blogs, in particular, result in that one on one equation where, for instance, one is attempting to meet with the thoughts of a blogger like Bill. It's that experience of confronting an intelligent mind and attempting to use one's wits to take in what is being presented. It's almost an intelligence test, in a way.

And, no doubt, it's this attitude that is resulting in my blank reaction to my understanding of the more generic social media sites. The experience of talking to strangers in a formal setting without an actual subject governing the interaction isn't something that makes sense to me. Clearly, the popularity of SM indicates I'm in a minority, and that the MySpace experience is something humans are longing for. What strikes me as frivolous is hitting someone else as an important form of human communication.

However, my whole feeling about this changes the moment a subject I am passionate about is setting the tone of the situation. Your reference to the green movement is particularly apt. Put me in touch with someone attempting to save polar bears and I'll talk my head off, because I'll be attempting to find a solution to something that grates on me, all the time, in real life. I'll be coming to the situation filled with intense emotions, a goal, aspirations and a strong wish of communicating with people who may be involved in important, meaningful work. When Social Media provides this kind of interaction, I really get the point of it. When it's talking about how many times one changed one's shoes that day...I've got other stuff to do.

In the end, I guess who you build for is what counts. If your demographic yearns to save the planet or compare recipes, it's all part of the human experience. Neat topic.

Edited by SEOigloo, 06 November 2007 - 12:27 AM.

[DCrx]

Are we not designing or even marketing for the human experience anymore and if so, why not?

Nobody knows how.

With usability testing, you have some fairly well thought out procedures. If you have a state of poor usability, and want to move to a state of better usability you can show a video of users having specific difficulties. When you're looking at analytics, you can at least see people are coming to a site but not buying.

With UX, there's no "there" there.

Case in point. The airline industry is 1) At the worst state of customer satisfaction since 1995 2) Flying with the fewest empty seats in years

The UX is awful and the key to success is being one half smidgen less awful than the competition. The point is nobody is eating the industry lunch on the basis of UX. When someone like Song airlines tries a different "user experience," the market hands them their head. This quickly shows management UX doesn't pay.

And unless you can show a competitor who's eating everyone's lunch, there is no incentive. If you can't say, this and that specific element needs changing, and then show a marked improvement in profit to management, nobody cares.

And UX largely can't. While you can define a task and do a task analysis, you can't get your head around an experience. And so you can't do an experience analysis. Nor can you do an experience test. It's all indirect inference and guesswork. That doesn't have the substance to get a well entrenched management to change.

UX is anything you want it to be. Web design is an "experience." Sites that allow you to vote a post up or down are "social." Why? Because nobody can say they're not. Apple has better user experience? Have UX explain how or why. Have UX predict the next Apple misstep. Explain the UX counter strategy for an Apple competitor.

By this line of argument, you can say an ant farm is equivalent to New York.

I'm practically the only one on the planet who can point to a methodology or test procedure that isn't warmed over usability testing. Believe me that until you can show a test, a methology, and that this design change produces that bottom line result, nobody in the marketing department is going to pay attention.

All they will do is keep doing what they've been doing, slap UX on it and ask for a pay raise.

I think half the allure of saying you're in UX is that nobody can say you aren't. Is the user engaged? Sure, why not. Is the user experience improved because of a specific recommendation? Whatever you say.

UX people want to get paid for doing something nobody can question or refute. If you can make a general idea sound like an implementable specific and get paid for it, that's a good job.

Edited by DCrx, 06 November 2007 - 07:37 AM.

[Autocrat]

I've always found that when you reach a certain point - it becomes pure guess work.
The reason is the time and effort involved in actually obtaining object data from a subjective source is damned hard.

For a demonstration - who has ever created a questionnaire (feed back form, user reports, focus group write-ups etc.)?
Did you ask open or closed questions.
When doing closed questions, where they simple A/B, or multiple choice?
If providing A/B questions - did you supply an even or odd number of questions?
If multiple choice, was there a limit on responses?
When doing open questions, how long was the question, and how specific?
The real crippler is - how did you phrase the questions?

As soon as you try to understand somethig as varied as opinion and thought - you run smack into a big black hole!
Even if you ignore that problem, you face interviewer bias if a human conducts it, as well as creator bias in the way that things are phrased, asked or the options provided.

You often end up with polarisation of questions, msot of it geared towards specific answers (whether you plan to or not, human nature dictates that we aim for the favourable - our favourable, not theirs!).

Then you haveto actually get people to respond.
You loose 12% on average due to neglect, lack of thought or simply auto-fill behaviours.
Of the remaining 88%, there is likely to be 8%-14% that are bias anyway due to the last experience - the extremes.
There is likely to be 20% that are "middling" - almsot all response atake the middle option, non-commital and of no real value - they simply answer "safely".

That gives you a working, fairly reliable 54%-60% --- how do you identify the "real" responses from the other types?


So bang goes the research option, as you are almost gaurenteed to have a large block of data that is simply "wrong".


The alternatives include following trends, following instincts, being inventive or basing on experience.

Trend following can result in Boom/Bust scenarios... as trends falter, or worse yet, simply disappear in a matter of days.
Worse, there are certain sectors of all markets that refuse to follow sheep and look for the more versatile and original suppliers/doers etc. (These consist of gold hunters, rainbow hunters, alpha types etc.)

Instinct following may appear to be a good chooice - but needs a fairly direct approach, with solid goals... and a lot of hindsight as well as foresight. It also realies heavily on the insticts being "correct" - that meaning the isntincts haveto be somewhat in tune with the masses/majority/money holders.

Being inventive is a good option for some avenues, but for general items it often proves to be dangerous and can cause drastic extremes... almost everyone loves it, or almsot everyone hates it.
The market is also largely made up of "sheep" - so if it's not already in trend, or simialr, a fair share won't like it.

Basing on experience would be the better option - but how many people from an airline company fly extensively... on other airlines as well as their own.
Of those, how many will actually be objective?


So, considering all of those - designing for experience is a major pain.
you cannot trust almsot half the results.
You cannot rely on others for correct details or non-bias views.
You cannot rely on yourself for the same reason as others.



At the end of it, you have to go for general, well known human nature and social responses.
Work up the Pyramid... cover all their needs, attempt cover their wants and if possible cover their desires.


The only real thing you can do is to try applying a little variable logic to mutable physical/mental/emotional aspects of humans in general.



So, after all that winding... no, we are not design for experience.
We should be designing from experience and common sense, whislt doing our damnest not to be too objective/subjective.

[bwelford]

This thread seems to have taken a very downbeat view of all this. If you fly by WestJet in Canada or, I believe Southwest Airlines, in the US, you know you're enjoying yourself much more than on other airlines. That can be measurable. Both airlines are very successful so doesn't that show that good user experience is what it's all about.

The key point is to see everything from a customer perspective. Try to live it like a customer. I believe that user tests and customer surveys, if designed well, can help you improve. However by the offerings from certain companies and agencies, it's quite clear they never do any of this.

Have I missed the point? To me it's a no-brainer that you will be more successful if you do things that customers like.

[DCrx]

You could say the exact same things about Qualty. Yet there is a thriving market for quality management, where actual methodology is supported with fairly concrete data.

And the objections, at the time, were quality is subjective and a black hole. So Demming took the ideas to Japan, which promptly started eating Detroit's lunch. Which, begrudgingly, gained Demming a receptive audience back in the states.

No lunch eating. No audience.

Let me introduce you to one small element which would be game-changing for the UX community: Kansei engineering.

Kansei Engineering (Japanese: 感性工学 kansei kougaku, sense engineering) is a method for translating feelings and impressions into product parameters, invented in the 1970s by Professor Mitsuo Nagamachi (Dean of Hiroshima International University). Kansei Engineering can "measure" the feelings and shows the relationship to certain product properties. In consequence, products can be designed to bring forward the intended feeling.

The objections against Kansei are just cut-and-paste jobs -- removed thirty years -- of Demming. First off, it is totally transferable and understandable to western firms ...with case history to back it up.

What's standing in the way of rapid adoption in America? Believe it or not the UX movement. I feel confident predicting you will not see Kansei research -- or anything like it -- at DUX. My guess is I could call up every leading UX expert and they wouldn't know what Kansei was, much less any of the procedures.

But Kansei and a wide variety of other techniques -- taken up by large firms from Mazda to Microsoft -- are there. It seems to me a well defined procedure and comprehensive methodology ruins the UX party.

Both airlines are very successful so doesn't that show that good user experience is what it's all about.

First off, there are a couple arguments. The accountants and analysts argue pensions and other sunk costs are lower on a range of industries, including SouthWest. Second, nothing about satisfaction ratings needs a fancy new term like UX. You can have this done by traditional marketing, and cancel seminars like DUX.

If UX wants to gain validity it has to show something different. It can start with the inconvenient truth that large chunks of those who say they are satisfied defect. Next it can suggest changes which change the numbers. And I would strongly suggest they be much more compelling data than satisfaction surveys provide.

you know you're enjoying yourself much more than on other airlines. That can be measurable.

This is like kissing a girl in the dark. You know what you're doing, but you don't get street cred because nobody else does. If it's measurable, measure it and show your methodology (a likely forum would be DUX -- if they'll have you).

I've shown you mine. UX ....show me yours.

Edited by DCrx, 06 November 2007 - 10:50 AM.

[Autocrat]

I completely ignored the whole "USer Experience" label, as it seems to be jsut that (yet another group of folk attempting to create a job position and title to earn warrant charging more money).


...bwelford...
I wouldn't say it was down beat, merely knocking rather large holes in the idea of aiming for experience.
The resutls are to vague and can, as always, be easily misinterpreted.
I'm more than tired of certain types of folk grabbing data and waving it around "this says XYZ".
It doesn't... it says exactly "1-2-3" - from there, I can make it say anythign I damn well choose.
It is how those figures are obtained that give them meaning and credit!


...DCrx...
The Japanes naming of things hit the UK around 5 to 10 years ago... and is still gaining support... which is hysterical as all it really does is apply a nice foreign label to thingsthat staff and workers having been suggesting/doing for years.


At the end of the day, going by results is fine. Going by experience is fine.
Going by both is much, much better.

If it helps make it clearer (as I appreciate I may be over-laying the point)...
All of us in this community can look at a single apple.
Then we can all sit down and describe our impression of that apple.
How many of use will describe it the same way... even loosely?
An extimated 30%-40% may use the same two or three word couplings - otherwise the details will differ quite abit.
Yet all we are doing is descibing a sodding apple.
Now think of all the differences in Flights, websites, etc. - many more differences, many more "impressions" and "perceptions"... and it is those that make "experinces".




I'm not even going to touch the social media side, as I think that is a rather different topic (Or at least, has many differing factors rather than just Experience/Perception)

[DCrx]

To explain how this works, which is where what I'm talking about with desirability differs from UX, take a kiosk for selecting styles of eyeglass frames.

A company wanted to put their whole selection, some 10,000 styles into a easy to use interface. The problem -- customers use purely subjective emotional language like sporty, fun, professional.

These terms mean nothing to engineers.

What Kansei allows you to do is isolate a specific criterion, in this example the relationship of the user's eyebrow to the frame, and connect these measurements to the words customers use. One set of measurements corresponds to "sporty," for a male, female, young adult or older person.

You can argue all you want with the methodology. You can run through the procedure and say, "Here is what it says to do." Usability offers something similar. You can be just as dismissive of a usability finding, but you can also run through the procedure, get a recommendation, and put it to a test and get a result. And that result will either bolster or undermine the usability person making the recommendation.

My point is UX practitioners can't be wrong. There is nothing to be wrong about. If they like it it's a "good experience," and if they didn't then not. That's more like what graphic artists say is good design and rather unlike usability practitioners.

UX people can't be bothered to point to those parts of an experience under the designer's control. They don't say this is good experience for demographic one, bad for demographic two, but if we redesign these ten things the experience will improve for both demographics without losing you business in either.

Also, this doesn't answer what Nielsen talks about as user opinion being three levels of abstraction removed from the truth. Again, if you can ask a user something, and they just tell you the UX level is good or great, do you just trust them? We know that is misleading.

Going back to the apple, if 50% of users say it's good and 50% say they had a bad experience, how do you account for this finding and more importantly what action do you take? UX doesn't seem forthcoming with a ready answer.

There is nothing about UX. It's an umbrella term to give fragmented sub categories some semblance of coherency and unity without proposing a unifying theory or methodology. It's a mental sedative designed to stop turf battles.

Edited by DCrx, 06 November 2007 - 12:47 PM.

[bwelford]

I'm not all that hot on Jakob Nielsen, DCrx, but in this case I think he has it right. Don't listen to users, watch what they do.

It seems to me that if we put the emphasis on user tests, we will be closer to getting it right. Could we not regard the user test as the fundamental standard? Of course it has to be done with suitable precautions to make it valid. Prior to having a prototype available for user testing, we could even do a mental simulation of how a typical persona might achieve their goals with the device or service we are about to offer.

Does that begin to bring enough science into the process?

[DCrx]

Does that begin to bring enough science into the process?

Yes. And the only issue I have is "enough science."

As you and I both know, you're never going to be able to convince everybody an idea holds water. Some people think the space missions were faked and the earth is really flat.

You can dismiss anything, out of hand, including everything in interaction design. The user test is a handy threshold. Can we outline an experiment. Can we gather evidence which will support or detract from the premise of the experiment.

It is a pretty low hurdle to meet.

Edited by DCrx, 06 November 2007 - 02:33 PM.

[bwelford]

I agree, it may be a low hurdle. On the other hand, I think that Pareto-type Law comes into effect here. If you put 20% of the total effort you might put in, you may well achieve 66% of the total benefits there are to be had. I've seen websites where if even one user had looked at it, there would have been incredible lessons to be learned from his/her interactions.

[Ron Carnell]

Can we outline an experiment. Can we gather evidence which will support or detract from the premise of the experiment.

It is a pretty low hurdle to meet.

You left out an important criteria, I think. To be scientific, your experiments and evidence have to also be repeatable. That, and the vagaries of human nature, will raise the bar considerably, I suspect.

Poor user testing, in my opinion, can often be worse than no user testing.

What makes that scary? When tested against the crucible of repeatability, all user testing is poor.

[Autocrat]

Thank you ...Ron Carnell...
You sum it up nicely - as soon as you start tackling human perception and reaction, things get messy quickly.
Then we are going a step further, adn asking for their responses to those things, and thier possible reasons.

To varied - the results obtained would be humongous in the variance levels.

You would then ave to resort to the same methods used in sociology and communications, and start grouping thigs - which beocomes completely subjective, as I can promise you that where I draw the line will not be where others do...
Also, peoples use of words generates different responses in the person conducting the questions...

Which is a more positive response:
Bearable | Acceptable
Technically, they mean that something "would do"... but Acceptable is viewed as more positive than Bearable... (does that make sense?).


You also forogt another prime requisite of making something scientific - you need a "control" or natural subject - how the hell do you decide the "normal" response?
(That is the main reason the pseudo sciences stay as pseudo!)

[iamlost]

I have re-read the limked column and the comments here perhaps a dozen times.

And simply can not get my mind to accept 'user experience' as a discipline. Each and every user is different. Therefore each and every user has a different experience. Possibly each and every engagement. It is almost atomic in it's specificity. A discipline, especially a social one, by definition, is about generalities. Thus I find myself agreeing with Mr. Jacobson that, to have a viable field of study, we need to generalise 'user experience' to the 'human experience'.

And then there is that word 'experience'. Put aside the world of measurable quantifiable physical reality and grasp that of transient emotional response and imperfect memory. My initial reaction was: fortune tellers and psychics in business garb!

With DCrx weighing in (I continue to learn much of value from your comments and site - thank you) I must take the concept (if not DUX the organisation) seriously.

When I remember some of the times (in the B&M world) customers told how 'wonderful' it was to do business with us the reasons they mentioned had never been implemented for their benefit at all. The low sloped ramps and auto-opening doors had been installed for ease of moving equipment dollies - but we were happy to take credit for being 'accessible' before it was popular. The separate child minding room off the showroom was to focus the parents, especially mothers, on our sales pitch but again we were happy to take credit for being family friendly. We built a business that was 'known' for being a great experience in a rather blah brusque industry (HVAC) while only making 'our' job easier. Perhaps a mirror image experience.

Off to read about Kansei Engineering.
Research, more research. Keeps the mind young, they say. Gives me headaches, I say.

[DCrx]

We only need minimal rigor for UX for one simple reason: We don't want to show up the rest of management as practiced. Management likes just a dash of methodology, and I don't see why UX can't oblige.

Usability and other broadly accepted practices are accepted, and as pointed out, there can be variances from test to test. Managers routinely employ all sorts of methods with questionable track records, from brainstorming to focus groups. So let's not hold UX or anything else to a standard well accepted techniques popular throughout the business world wouldn't pass.

The point is there are lots of methods out there. Microsoft's desirability toolkit. Prelec's information pump. All provide the jittery manager with a shotglass full of rigor to quiet the nerves. All these have a track record of multi-million dollar decisions, like Kansei and the Mazda Miata.

All have several decades of research history to their credit.

This is what experience engineering looks like, and it is several generations ahead of where UX is today.

Humans aren't logical, and that's why logic tools fail. You're not trying to get humans to somehow become logical enough for engineers. Instead you're trying to employ tools which dig a little deeper, get more honest, useful answers to questions.

Now for the big question is, where is the "better way?" Can we replace nebulous concepts with some concrete ones. Turns out that yes, you can.

One idea is customer sacrifice is different than customer satisfaction and correlates better with the percentage of customers who remain loyal. Pine and Gilmore, in looking for the gap between what customers really want and what they settle for is useful in locating an experience deficit.

What does an experience problem look like, and what are the business implications. Agree or disagree, Pine and Gilmore can at least tell you.

You can disagree, or outright hate the idea of an experience economy. However, when you're trying to get your head around the concept, sacrifice works and is at least an understandable concept. If you wanted to go out next week and "do something," you can have a course of action.

You can read about the types of sacrifice with examples in Customer Satisfaction Is No Longer Enough.

I submit that this way of thinking, while just scratching the surface is something you can discuss in the workplace without getting pushback. You can perform experiments, collect data and present a cogent argument that is in tune with what management likes to see.

I've really tried to understand where these guys are coming from -- even attempted to get interviews as content for my website. You would not believe the conversations I've had that just did not add up. So I put together all the stuff that did, and developed a coherent framework to tie all the ideas together. Each dimension has a methodology and a test procedure with a fairly long track record.

It's much more confidence boosting when you're talking to a manager about the engineering of the Mazda Miata then some of the more nebulous concepts. And of course, the technologists don't tend to roll their eyes when you talk about things this way.

Edited by DCrx, 07 November 2007 - 07:00 AM.

[bwelford]

Are we perhaps trying too hard to get the ultimate solution here? Perhaps anything that makes things somewhat better that can be repeated over and over again is a useful process.

[yannis]

Poor user testing, in my opinion, can often be worse than no user testing

.

On new projects user testing may or may not depend on the individual alone. What physics can bring to the social science of decision making is not a fully fledged and precise theory but a deeper understanding of an ingredient that has been misunderstood in the past. But we should study this ingredient well. It is the effect that one person has on another, the influence of person to person interaction. On the web this is even more important. One of the features of collective behavior arising from local interactions is that it becomes impossible to deduce the state of a global system by inspecting the characteristics of its individual components. The crux of the message here is : do not be tempted too readily into extrapolating from the psychology of the individual to the behaviour of the group.

When you got up this morning (paraphrasing Joshua Epstein) it never crossed your mind to drive on the left (but I do when I am in RSA or the UK!). At lunch you would never consider to eat your food bare-handed; without a thought you will use a fork. As individuals we are very suggestible to what we see and hear.We are acutely sensitive to even the smallest details of everyday life. We learn this type of behaviour very fast and early.

IMHO this aspect of UX or similar disciplines make any proper scientific study difficult.

Yannis

[DCrx]

For me, I have no problems with the "yes but it's not a sciiiieeence" crowd.

People still pay psychologists. One of the few vivid recollections from my intro psych course at college was answering the question, day one, of whether psychology was a science.

We all paid money for the course. Likewise, usability is taught at colleges. And, increasingly, so is a lot of the stuff on my site. Including for the purpose of this topic...

Dr. John Monberg
WRA 415 Digital Rhetoric
http://www.msu.edu/~...Experience.html

That's a user experience course, BTW. So I don't think my perspective is entirely off base.

So I don't get too worked up over it. My qualm is too much of UX strains credulity.

[Autocrat]

THe hard ship is in how to ensure the methods applied produce "valid" results.
By Valid, I mean fair, non-bias (as much as possible).
From there, you then have the seriously difficult task of coalating and computing the results into something useful (again, avoiding as much bias as possible).

you cannot get 100% - due to the sheer numbre ofpotentials and variances.
Definitions shift and change between individuals, as do the reasons for forming opinions.


It is doable, so long as people remain constantly aware they are using a very imprecise tool, and a methodology with some large holes.
So long as that is accepted, and the results are treated wit hthe care such fragile things deserve - then by all means use it.

Applying simple logic will work, high-end and deep logic is likely to spoil the data and introduce bias elemets.
Generalisation ad flexible grouping would be the key, imo, to keeping thins as safe as possible.
The other approach, used to help define Communications, is to ask open ended questions, then group the results by phrasing. A tedious and time consuming process - but one that will give you more "natural" groups than hard-lined definitions decided before the results come in.
Set the general goal/s, get the game in action, collect the results, then decide waht scores what when looking (rather than doing so before the game is played!).
Not easy, but seems to be a fairer approach.


I'm still not sure on the label though - it still "smacks" of mysticism and redirection.
But the general approach is somewhat feasable (I jus wouldn't trust most people to do the job - especially not anyone that makes claims to be a specialist in such areas).

[A.N.Onym]

For those interested, Human Factors has a webcast and a whitepaper on UX metrics. The whitepaper doesn't seem to be available from a personal link, so you can only get it from their main page:
http://www.humanfact...ds/webcasts.asp