Roger at 456bereastreet illustrates how to usefully enact a small scale usability test. I commented there; the expansion follows.
I think it’s important to stress how much usability norms can differ across different audiences and regions, and how much adaptability and approachability after publication should be more of a factor in determining requirements. A company/organisation/community with no prior experience of its audience will have no conception of the capabilities permissible, and so would have to offer the bottom line. It may turn out that that’s insufficient, because your majority audience turns out to be extremely competent and technologically advanced – they may desire the complexity, because they can use it.
See the (admittedly badly formed) article at The Register on Google and scope for some brief comments on the different perceptions that east and western cultures have of what the internet itself means, let alone usability of it.
Any designer should aim to follow the simplest approach possible, adding on optional extras as the user goes through the process, without complexifying things too much. Usability in such a case might therefore be reducible to a set of criteria, or dare I say it, standards.
But it can’t, of course, and we know why. Usability, and accessibility, are individual characteristics particular to each and every user, and so depend on a combination of user knowledge and education, and designer prediction and “leeway” to allow a vaguely balanced experience. (let’s exclude perhaps one of the least usable sites on the web, Jakob’s, which fails not by structure but by design)
All of which brings me to: usability and accessibility studies which involve “real life” users are immensely important until the designer gets it. At that point, unless they pay no attention to web trends and societal change, they have a reasonable – while not expert – understanding of what to provide.
We can try to appoint a representative council of “disabled people” to test our websites, but have we considered the depth of experience that different blind people have with different screen readers? Have we considered the lack of knowledge of basic computer usage, that many disadvantaged people may have, or the misinterpreted functions of lacklustre browsers and operating systems (alt = tooltip being a relatively minor one)?
Each person is an individual. A person is impaired by the barriers placed on them; as a designer, it is therefore critical to ensure that a website contains as few barriers as possible, without even considering the type of user who may stumble across it. It isn’t about testing whether a blind person with an advanced knowledge of JAWS can use a site: of course they can, they’ve persevered for the past 8 years trying to make sense of it all, so they can work out your silly little code. It’s about the baseline, the average user who has a very slim understanding of the internet, of the way a browser works, of the way a computer works. We have a project which looks at the way people from different backgrounds may face challenges using a VLE, but it can apply equally to websites.
In fact, in a lot of cases, you may find that disabled users are more competent than their counterparts, purely because they’ve had to try harder. That’s not a problem, but worry less about the specifics of how a particular screen reader may respond to a certain type of list, and worry more about whether the harsh black on white text you have will cause people with specific learning difficulties, people with a tendency for migraines, and people with even marginally poor vision, to react badly to your site and may even force them away.
Furthermore, reasonable is perhaps the best stage to be at. Nielsen may have an excellent understanding of usability, but he has no idea how to implement it. Joe Clark has a fabulous understanding of captioning and many accessibility issues, yet he still frightens everyone off by being Joe Clark, hence prohibiting access. Experts are absolutely necessary, but not in proliferation; here the jack of all trades is more or less enough.