I have just discovered an online survey being run by the National Trust about the iconic Lake Burley Griffin in Canberra. It is very welcome that the National Trust is embracing the web to engage its community and it seems a well intentioned initiative but I wonder how well thought through it has been. Specifically the choice of a survey for this particular project.
The survey begins with a statement that it will take about 30 minutes to complete. If that doesn’t put you off the first page is a whole series of personal questions about your demographics that are entirely irrelevant to the issue. I know these are there in the name of getting a credible stratified sample but really – who are they kidding? – how can the group who would go to the National Trust website and volunteer to undergo a 30 minute survey be in any way representative of anything other than the small part of our community who might be prepared to do this? Showing they have different demographic characteristics doesn’t really change the fact that this group will never be representative of the community as a whole.
To get the wider community participating people need to be able to express their views in a faster way and to focus on what is important to them.
“I think the facilities in Weston Park for children’s play should be upgraded”
“Cycling facilities should be improved”
“Switch on the giant fountain thingy more often, it never seems to be on when I visit” (National Trust please take note of that one).
By allowing the community to express their views in a quick and painless way I believe that the survey will become more representative of the community because you won’t have to be a passionate Lake Burley Griffin user to take the time to join in. Of course you won’t be able to prove it but does that really matter if the aim is to get the community to join in and provide input?
This issue of stratified sampling comes up a lot. The online forums we run at Bang the Table don’t even attempt stratified sampling. We believe that gathering the information required to do this is a serious impediment to people participating and so while you may have information about the age and socio-economic status of the respondents you will also exclude some people by the very fact that you ask these questions.
Also, and this might just be me, do you ever take the time to make sure you respond to the demographic stuff accurately? I just click through it. Maybe others take more time but what good is demographic data that cannot be verified?
I would not begin to argue that stratified sampling should not be used in surveys. I think it should and that whenever possible it should be backed up with sampling of value cohorts within the different demographic groups. Demographics speak to age, race, how rich we are and how educated we are but they don’t speak for our value sets and these are just as relevant as demographics.
What I would argue strongly is that surveys are the wrong tool for a consultation like ‘tell us what you value about Lake Burley Griffin’. If you want the community to participate you need to provide easy to access means of engagement that encourage maximum participation and allow people to talk about the aspects they want to discuss not to wade through a 30 minute survey hoping their issue might come up at some stage.
For those who are interested my issue with the fountain may be in there somewhere but I’m afraid I gave up after 6 minutes when the phone rang…
Photo Credits: Ian Sane
Thanks for getting all the way to bottom! Subscribe to our monthly digest newsletter if you’d like to be kept up to date about community engagement practice globally. Take a look at our two product websites: EngagementHQ if you need a complete online engagement solution, and BudgetAllocator if you need a participatory budgeting solution. Or get in touch if you have a story idea you think is worth sharing.