The number of responses needed for statistically valid customer survey results

It is critical to know what our customers think of living in the homes we build for them and what it was like to be involved in the design and construction process with us. In the case of survey results, this is a lagging indicator but still a track record of how we have done and it is powerful data. The more accurate the data and the faster we gather and act upon this data the more quickly we can listen, evaluate, correct anything that went wrong, prevent it happening again and seek further opportunities to improve.

There are a number of factors to address to ensure that the surveys have an impact.  Just one for example is ensuring that we actually ask the right questions in the first place, more is not better!  Another is ensuring that your survey results are statistically valid, that is, the responses (sample) are of a statistically representative size.

While getting the most surveys back is the goal, in some cases you don’t need 100% or even 50% to be valid! As the size of the group you want to survey increases, the sample size (the number of surveys returned from the group) falls. You can view this as diminishing returns. At a point you will get the same result if you have 25% response rate as you would with 90%. While we want to get as many surveys returned as possible, the statistics show us that at a certain point we can stop using our resources in getting more completed surveys back and instead focus our time on something else instead! Indeed, we can cause more problems by ‘pushing’ for returns, research shows that satisfaction scores go down the more the respondent feels ‘coerced’ into completing a survey. This can simply be from getting too many reminder emails and certainly from ‘personal reminders’. So some out there may be able to use the time and resources they are currently using to hunt down survey responses to study the data more by using correlation or regression, implement more improvements or spend more time with that customer going over their home plan.

In the columns below, population is for example the number of homes sold in the past year, the sample is the number of households you need to have respond (complete and return) your customer surveys to be statistically valid.

This is at a 95% confidence level (how sure you can be with the information provided) at a +/- 5% degree of accuracy (margin of error- confidence interval). Or that your results will be within 5% of the correct answer 95% of the time you run your survey.

Population       Sample (Responses)

10                    9

50                    44

100                  79

150                  108

200                  132

250                  151

300                  169     

400                  196     

500                  217

600                  234

700                  248

800                  259

900                  269

1000                277

1500                305

2000                322

3000                340

3500                346

4000                350

4500                354

5000                356

10,000             369

20,000             376

30,000             379

40,000             380

50,000             381