Testing With Real Users

What "we tested it" actually means

Automated accessibility checkers catch perhaps a third of WCAG failures — the ones that can be read from the code. The rest only surface when a person who uses a screen reader, a switch control, or high-contrast mode actually tries to complete a task. "We ran it through an automated tool" is not testing; it is a starting point.

Meaningful testing with disabled users is not a single sign-off session held two weeks before launch. By that point, the cost of fixing a structural navigation problem or a form that breaks with keyboard-only input has multiplied several times over. Involvement needs to happen at the wireframe stage, again at prototype, and again when the build is close to complete — not because each round is a compliance ritual, but because different stages surface different problems.

Who participates matters as much as when. A panel of screen reader users tells you nothing about how the interface performs for someone using voice control, or for a user with a cognitive impairment who finds multi-step processes disorienting. Recruit across impairment types, and recruit people who use their own devices and their own assistive technology — not a lab setup where everything is configured identically. Real use is idiosyncratic. Testing should be too.

What organisations most commonly get wrong: treating user testing as a validation exercise rather than a discovery one. The purpose is not to confirm that the product works; it is to find out where it doesn't. That requires genuine tasks — "book an appointment", "find the complaints process", "update your direct debit" — not guided walkthroughs where a facilitator smooths over every stumble.

Pay participants. Set a rate that reflects professional expertise, not voluntary goodwill. The knowledge that a screen reader user brings to a testing session is specialist knowledge. Treat it accordingly.