The number everyone knows and nobody checks
Ask any product team how many people to test with and you will get five. It is the most successfully transmitted finding in the history of user research, and it is transmitted with the crucial qualifier stripped off.
Jakob Nielsen's 2000 article, building on a model he published with Thomas Landauer in 1993, estimates that five participants reveal about 85 per cent of the usability problems in an interface. The model assumes each participant independently finds a given problem with some probability, averaging around 31 per cent across the studies they drew on, and the curve of cumulative discoveries flattens quickly from there.
The qualifier is that this describes one homogeneous group of users. Nielsen says so in the article. It is simply not the part that travels.
Why the qualifier is the whole finding
The maths works because the five participants are drawing from the same pool of problems. They are attempting the same tasks with roughly the same prior knowledge, so their discoveries overlap heavily, and overlap is what makes the curve flatten.
Put an administrator and an end user in the same set of five and that assumption breaks. They do not attempt the same tasks. They do not share a mental model of the product. A problem that stops one of them cold may be invisible to the other, so their findings barely overlap and the curve does not flatten at all. You have not run one study with five people. You have run two studies with two and three.
Five is not a budget. It is the point where a specific curve stops repaying you, and there is one curve per group of users.
I work mostly on enterprise SaaS, where this bites hardest. A security product might have an analyst who lives in it daily, a manager who opens it weekly for reports, and an administrator who touches it twice a year to add users. Those are three curves. Fifteen sessions, not five.
What five users does not tell you
Even inside one segment, the number answers a narrower question than teams think.
- It counts problems, not severity. Five participants will surface most issues, but a problem that only one person hits can still be the one that loses an account. Frequency in a test is not importance in the market. 2. It says nothing about how common a problem is in your user base. Discovery and prevalence are different measurements, and only the second one needs a large sample. 3. It does not cover rare paths. Error states, recovery flows and edge-case data barely appear in five sessions unless you deliberately construct tasks that reach them. 4. It is not a benchmark. Comparing two designs, or claiming a task-completion rate, is a quantitative question that needs a quantitative sample. Five people cannot give you a percentage anyone should act on.
The trade that actually matters
The genuine insight in the 1993 model is not the number five. It is that testing the same design with twenty people is a worse use of twenty sessions than testing three iterations with five people each.
The second design is different from the first, so it has a fresh set of problems and a fresh curve. Spending the whole budget on one round buys diminishing discoveries about a design you already know is wrong. Spending it across rounds buys three flattened curves.
That is the finding worth defending in a planning meeting, and it is the one people drop in favour of arguing about sample size.
How to size a study without pretending
- List your genuinely distinct user groups. Distinct means different tasks and different prior knowledge, not a different job title on the same workflow. 2. Budget four to five participants per group, per round. 3. If that number is impossible, cut groups rather than thinning them. Three solid sessions with analysts beat one each with an analyst, a manager and an admin, which tells you nothing about any of them. 4. Plan more than one round before you plan a large one. 5. Judge severity separately from frequency, in a conversation with the people who own the outcome, not by counting how many participants hit each issue. 6. Reach for a bigger sample only when the question is quantitative: comparing designs, sizing a rate, or setting a benchmark.
Where small studies quietly fail
The failure mode I see most often is not sample size at all. It is that the five people were recruited for convenience, so they resemble each other and the team in ways nobody wrote down. Colleagues, friendly customers and anyone already fluent in the product will find a narrower set of problems than the model predicts, because their discovery probability for the problems that matter is far below the average the model assumes.
That is a different failure from testing too few people, and adding participants from the same convenient pool does not fix it. Related: your own team is not a sample either, which is the argument I made about where dogfooding stops working.
The point
Five is a good number. It is a good number for one group of users, in one round, when you are hunting for problems rather than measuring them, and when the five people actually resemble the people who will use the thing.
Strip any of those conditions and the number stops meaning what the citation implies. The productive move in a planning conversation is not to argue about five versus twelve. It is to ask how many genuinely different users this product has, because that question changes the answer far more than the sample size ever will.
Frequently asked questions
- How many users do you need for usability testing?
- About five per distinct user group, per round of testing. Nielsen's model estimates five participants surface roughly 85 per cent of usability problems within one homogeneous group. If your product serves several genuinely different user types with different tasks, you need four to five for each of them, because their findings will not overlap enough for one curve to flatten.
- Is the five-user rule still valid?
- The underlying model holds for what it describes: finding problems within one group of similar users during qualitative testing. It was never valid as a total sample size for a whole product, as a way to measure how common a problem is, or as a basis for comparing two designs. Those are quantitative questions that need larger samples.
- Should I test with more users instead of running more rounds?
- Run more rounds. The original 1993 work makes this point directly: three rounds of five participants on three successive designs finds substantially more than one round of fifteen on a single design. Each redesign creates a new set of problems, so a fresh round starts a fresh discovery curve rather than continuing an exhausted one.
- How many participants do I need to measure task success rates?
- Far more than five, usually twenty or more per condition depending on the precision you need. Discovering that a problem exists and estimating how often it occurs are different questions. Small qualitative studies answer the first well and the second not at all, so a completion percentage from five sessions should never be reported as a metric.
Sources
This thinking, applied
Share this
