Nanaimo-Info-blog: Nanaimo Ipsos Reid 2012 Survey

Ipsos Reid 2012Citizen Satisfaction Survey
The results of an Ipsos Reid Citizen Satisfaction Survey conducted in Nanaimo in May of this year revealed some interesting results……..or did it?

The reason for wondering about the value of the survey is based on sample size and the methods used to craft and gather the information. In her presentation Ms. Knaus an Associate Vice President, Ipsos Reid Public Affairs said the survey questioned only 300 people and according to her had a +/-5% margin of error 19 out of 20 times. This is more or less the same number of people who actively participated in our recent strategic planning study. The difference being, that the Ipsos Reid survey attempts to control the demographic of the people they are questioning.

One reason I would question the validity of such surveys is the fact that companies such as Ipsos Reid are in the business of selling surveys, that is their stock in trade, that is what they do. So ask yourself, do you think that more municipalities will buy their product if the results indicate a good level of citizen satisfaction? Being just slightly cynical I suspect a company bearing really bad news would wind up being the messenger that gets shot.

In his article ‘How To Manipulate Citizen ‘Approval’ Surveys’ author Robert Gerst explains how a survey can be designed from the beginning to guarantee a higher level of satisfaction than what actually exists in the community. In addition to carefully crafting the questions he also reveals what he calls a trade secret which will yield a high level of satisfaction.

He says:
This is the best trick of all and a trade secret. If you want to guarantee positive results on your survey, use a small, five point scale, and report the top two box scores – where a score of four or five is recorded as ‘positive’ or ‘satisfied’. Where’s the trick?

It’s in the phenomena called positive response bias. People tend to answer to the positive, even when their attitudes are neutral or slightly negative, answering four when three better describes it. A tally of ‘satisfied’ respondents, therefore, includes people ‘neutral’ or ‘slightly dissatisfied’ ensuring an inflated positive result.

Check this out for yourself. Download a citizen satisfaction report and see if it doesn’t rely on a top two box score summary on a five point scale. By the way, the smaller the scale, the stronger the bias. Citizen satisfaction skyrockets with four point scales.

Note: the survey that Ipsos Reid used in Nanaimo this past May did in fact use a four point scale to gauge satisfaction to different questions.

Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest