Measuring online citizen engagement: Setting effective metrics and targets
In building the latest version of the EngagementHQ software we not only thought about new tools and ways to engage the community, we also watched the ways our clients had been using the reports and set ourselves to thinking about how we could build a set of metrics for target setting and the measurement of results that will remain relevant as we add more and more functionality to EngagementHQ.
Things have changed a lot since we designed our old reports. You can now get information from your community using forums, guestbooks, a story tool, interactive mapping, surveys, quick polls, submission forms, a news feed with discussions or the QandA tool. You can provide information to the community not just through library, dates, photos and FAQs but also using videos, link boxes and embedded content from all over the web.
Our old reports could tell you that 600 people had viewed the documents and it could tell you that 70 people had read the FAQs but you could not tell if they were the same people so you didn’t really know how many people had accessed information through your site. Generally we used those who had viewed documents in the library as a proxy but as time goes on our more engaging clients are communicating less and less through documents and more through other channels.
Similarly, whilst registrations were a good proxy for engagement (why else would you sign up?), it was failing to keep pace with the technology. You can now configure all our tools to require sign up or to be exempt from it these days so the proxy doesn’t hold. Moreover, many of our clients bulk load groups into the database and therefore inflate the registrations number.
What we came up with was a simple solution. We would calculate Aware, Informed and Engaged cohorts in the reports.
Aware – a measure of the number of people who have visited your project;
Informed – a measure of the visitors who have clicked to access further information resources, to learn more;
Engaged – a measure of the number of people who have given you feedback using any of the means available on the site.
Our reports do provide breakdowns of these numbers – for instance you can still see how many people have viewed documents and which documents have been viewed; but we think that the Aware, Informed and Engaged cohorts will provide the benchmarks for target setting and the measurement of engagement outcomes in the future.
If your project is about getting information out or about exhibiting your intentions (a draft strategy, a development plan etc) then you should be targeting a high number in the Informed cohort. If you find you are getting high aware numbers and low informed numbers you need to make your information clearer, more attractive and consider using more engaging means to get it across (don’t make your community read a lengthy document, break it down, explain it on a video, try an infographic).
If your project is collecting views, vision, testimony or stories then you are targeting a high Engaged cohort. If you are not getting enough results through work on your call to action – making it clear why people should respond. Consider how you are asking people to engage – perhaps invite them to tell their stories as well as filling in a survey form. You might also recognise that it’s generally unusual for more than about 10% of visitors to engage and therefore work to publicise the project more to drive up the traffic.
We have plans to build all sorts of enhancements to EngagementHQ’s reporting over the coming years but there is no intention to move away from the Aware Informed Engaged cohorts, these should be at the heart of all target setting and measurement of engagement outcomes.