How Advanced Text Analysis Can Make Open-Ended Digital Engagement More Approachable
To help you extract more meaning from written responses in less time, we’ve released three powerful upgrades to Text Analysis reporting this year. While each new feature is powerful and exciting on its own, using them in combination with one another creates an interactive opportunity to explore written comments at scale. Whether you’ve chosen a Forum or an Ideas tool for open-ended engagement, every engagement tool that collects written feedback through EngagementHQ will be available within Text Analysis reporting.
Text Analysis reporting now reveals who said what at a deeper level, while uncovering how each person feels about a given topic.
- 👩🏫 The who: Demographic filtering allows you to visualizevisualise who participated and isolate responses based upon registration questions.
- 💭 The what: Advanced search and comment tagging allow you to quickly organizeorganise what was said to identify trends and themes.
- 💌 The feeling: Through artificial intelligence, Sentiment Analysis will automatically assign positive, mixed, neutral or negative tags to each comment and deliver an interactive summary.
Combining two or more of our new text analysis features virtually creates a form of cross tabular analysis to help uncover the story hiding within large datasets. Together, you’ll be able to isolate what a specific location or neighbourhood thinks about a particular topic and visualizevisualise how they feel when contrasted with the broader community.
To tell the story of what these features achieve and how they work together in practice, let’s walk through a hypothetical example. Let’s say you received 500 comments to the broad question: “What should the city build to improve or expand recreational facilities?”
You may be daunted by the sheer volume of responses, yet feel the responsibility to produce valuable and trustworthy insights for team members in the form of a summary. You may be looking to close the loop with the community or use these insights to inform later phases of an engagement project. But you only have a finite amount of time available. Here are are some of the techniques you can use to deliver an insightful and trustworthy summary in the shortest amount of time possible.
As you begin to familiarizefamiliarise yourself with what people have written, you may notice similar comments and stumble onto a theme or trend. This is when our new advanced search functionality comes in handy.
On the first page of responses, you find three comments mentioning parks for children. At this point, you can decide to continue reading comment after comment to identify similar comments, or you could rapidly test your observation by starting an advanced search for “park.”
Let’s say your search narrows the results to 55 comments about parks out of 500 total comments. A quick scan through these comments might reveal additional words related to the same topic. Such as recreation, playground, or field. With these words in mind, add to your multi-keyword search to locate every comment that includes at least one of these words. You may notice some comments about parking, to eliminate these instances from this search, you could quickly add an exclude word for parking and quickly search again.
You’ve successfully performed your first advanced search, and now there are 105 results related to parks. From here you can decide to group all of these comments by checking “select all” and adding a comment tag for “Park” into the text field.
After tagging the majority of comments related to parks, you might want to know how many comments about parks also contain references for children. Locate and apply the comment tag “Park” from the comment tag dropdown menu on the right, and begin a new search with words related to children. In this case, 30 people wanted parks for children. To save this view of comments about parks for children, you create another tag called “Parks for Children.”
Rinse and repeat. Before long, your entire dataset will be coded for you to then dive deeper into who said what and how they felt. With a little practice, you’ll begin to fly through data at scale to deliver meaningful reports and summaries.
Demographic filters allow you to isolate comments by age groups, gender, location, interests, and more. Demographic filters are powered by site registration questions, meaning it is always available to you and participants will never have to answer demographic questions twice.
To begin, enable the demographic dashboard in the upper right-hand corner and take a look through each demographic category. What stands out to you? Which groups participated the most and least? Is your sample representative of the larger community, or are there overrepresented and underrepresented groups? Throughout the lifecycle of your consultation, you may visit your demographic dashboard to be proactive and ensure equitable representation across demographics.
Since our sample question was about investing in public spaces, you may want to draw insights by reading comments based on participants’ location. To view the ideas shared within a specific area, apply one location as a filter to refresh the list of comments. As the data refreshes, observe how the sentiment summary changes. Observe the type of tags that show up most frequently. Apply a comment tag to view the comments made by a group on any specific topic. This type of analysis is effectively a soft form of cross-tabulation and helps extract meaning for summaries and support your conclusions.
Since different age groups may lead different lives, you could analyse and report on the top themes within each of the age groups. For instance, those in their 20’s might want bike lanes, parent’s might want parks, and grandparents might want more gardens. But what idea do they share the most in common? With your comments tagged thanks in part to advanced search, the answers to these questions are readily available to you.
Tell me how you really feel?
Sentiment Analysis is an automated process powered by artificial intelligence that quickly reads and prescribes a positive, neutral, mixed or negative sentiment tag to each comment. Additionally, it allows you to confirm or amend the sentiment as you read through responses to ensure its trustworthiness. The sentiment summary chart will update according to your demographic filter or comment tag selections. The summary chart itself is also interactive, allowing you to isolate positive, neutral, mixed or negative comments.
Let’s say you’ve successfully identified popular ways to invest in recreational spaces and concluded the ideation phase. You then implemented a survey question or Ideas tool to vote on the 10 most common suggestions from your original question, narrowing the choices down to two options. To make the most of Sentiment analysis, you could then probe a more specific question asking “to improve our public spaces, should we build a new park by Lake Hefner with a community grill, dock, and playground?”
This type of question produces pointedly positive or negative comments for artificial intelligence to interpret easily. Sentiment analysis creates an opportunity to design questions that produce truly nuanced insights while being easy to interpret at scale. This should never replace reading responses altogether, however, it does turn text into data to provide valuable context.
With just a little bit of practice, you’ll be able to process more feedback in less time to uncover meaningful insights for use in summary reports. These features are quite powerful, interactive, and playful. You’ll be able to explore your date from various angles and quickly test hypotheses to create meaning out of obscurity. To take your analysis one step further, comment and sentiment tags will be included in an exported excel report.
Each new feature is available now for free to all clients and may be applied to any tool that uses written feedback. Advanced search and Demographic filtering are currently present on all EngagementHQ sites. Sentiment analysis may be enabled by visiting the EngagementHQ marketplace and clicking the button to activate it.