PowerPoint Presentation
1
How PublicParticipationCan Help Predict!Outcomes
2AgendaMarket Research 101Online Polling: A New World OrderPlaceSpeak Case Study: Transportation VoteCompare & Contrast: Polling & ParticipationHindsight is 20/204 Ways PlaceSpeak Can Help Improve Results Conclusion
3
How PublicParticipationCan Help Predict!Outcomes
I. Principles and Trends in P.O.R.What is public opinion research?Public opinion research is comprised of two distinct but related products market research and public affairs research4
ProductsServicesHabitsPolicyIdeologyVoting IntentionsCompaniesPlannersAgenciesPoliticiansResource &Development
Research companies perform surveys or interviews to assess public opinion. When they focus on products, services, or habits - that is market research policy, ideology, or voting intentions (polling), - then that is public affairs research.
Corporations use market research to sell products or services.Politicians, policy-makers such as planners, and agencies want to monitor public opinion so they can respond to sentiment.
5
How is public opinion research performed? Information is collected using either focus groups on a small scale On a large-scale information is collected using surveys. Focus groups are moderated and recorded in-person discussions between participants selected to represent the target group. They are expensive and they do not have sufficient sample sizes to generate real quantitative statisticsHow many how often etc But they provide lots of qualitative dataHow do you feel, what do you think etcPersonal interviews have similar strengths and weaknesses as a research method. To generate large-sample data, surveys are usually required. Surveys need to be standardized so that everyone gets the exact same experience answering the questions6
What makes a good survey?When it comes to surveys, the larger the sample, the more precision is permitted in the answer. A survey of 1000 people can yield a result with an error margin of about 3%, whereas a 2000-person survey can yield a result with an error margin of about %2. The larger the sample, the smaller the margin of error - but at some point the margin of error is negligible; you dont get more value by running a bigger survey7
But sample size per se has no connection with representativeness, or with the accuracy of results. The bigger the sample size, the more precise the estimate; but the more representative the sample, the more accurate the estimate 8
https://www.marketresearchcareers.com/marketresearchprosurvey2015.aspx
Typically there are four methods for administering a survey. In decreasing order of cost and reliability, they are:1. In-person interviews2. Phone interviews3. Mail-outs4. Online9
Online Polls Are Better Than Phone Surveys in Almost Every Way Based on my conversations with colleagues in the industry, rejection and non-completion on phone polls run to the 95 per cent range. This means, of the 10 people a polling firm might contact, more than nine aren't inclined to do the survey. What's left is a questionable mix of available respondents who often don't properly reflect the diversity and demographic differences of Canadian society.http://www.huffingtonpost.ca/angus-reid/telephone-polling_b_6958384.htmlAngus Reid
Online Polls Are Better Than Phone Surveys in Almost Every Way Based on my conversations with colleagues in the industry, rejection and non-completion on phone polls run to the 95 per cent range. This means, of the 10 people a polling firm might contact, more than nine aren't inclined to do the survey. What's left is a questionable mix of available respondents who often don't properly reflect the diversity and demographic differences of Canadian society.10
http://www.nationaljournal.com/magazine/who-responds-to-telephone-polls-anymore-20120719This is supported by published research on telephone polling. Contact rates have dropped by nearly thirty percent, and cooperation has fell correspondingly. Pew reported that its contact ratethe percentage of households in which an adult was reached at allhad fallen to just 62 percent in 2012, down from 90 percent in 1997. Of those successfully contacted, the cooperation ratethe percentage of contacts with an adult that yielded an interviewwas only 14 percent, down from 43 percent 15 years earlier. That means Pews overall response ratethe rate of completed interviews to the number of phone numbers dialedwas just 9 percent, one fourth of the 36 percent level from 1997.
In simpler terms, in 1997, a pollster could get a survey completed in 1 out of every two answered calls or connects. Today, that number is around 1 in six answered calls or connects.
Its harder to get connects, and its more difficult to keep people on the phone.11
http://www.nationaljournal.com/magazine/who-responds-to-telephone-polls-anymore-20120719So why the sudden fall off? Perhaps theres a correlation between the number of people that actually still have Land Lines in the home? Is there any coincidence that the number in this survey indicates about 32% of adults do not have a Land line at home while the previous survey shows that pollsters are achieving connect rates that are 30% lower?
Regardless of the coincidence, the fact is that internet accessibility is higher than land-line telephone installations. 12
http://www.statista.com/statistics/284206/north-america-mobile-phone-internet-user-penetration/
http://www.economist.com/node/14214847Telecoms operators are seeing customers abandon landlines at a rate of 700,000 per month. Some analysts now estimate that 25% of households in America rely entirely on mobile phones a share that could double within the next three years. Similarly, North American internet penetration is reaching an estimated 70% - with most metropolitan areas well in the 80% range.
13Online polls can be "more accurate" because people are more candid responding to a computer than a live voice in many cases.
http://m.washingtonexaminer.com/poll-wars-pew-says-internet-polls-often-more-accurate-candid-than-phone-surveys/article/2564575Online polls can be "more accurate" because people are more candid when responding to a computer and not a live voice in many cases. After all, a computer is non-judgemental.
There is some evidence that indicates the difference can represent just a small margin of error of only a percent or two But it could also mean to a very significant point spread of twenty percent or more.
This could be because when people are on the phone, they may just be inclined to complete the survey, without much thought to the result whereas if a person is online and completing the survey, they may be more inclined to have set aside the time to give each question a degree of deeper thought.14
http://www.pewinternet.org/2015/05/20/americans-attitudes-about-privacy-security-and-surveillance/pi_15-05-20_privacysecurityattd00/1But then, just perhaps another reason why people wont cooperate with telephone pollsters is trust. People dont trust a stranger on the phone as much as they trust online. It would appear that we are transitioning to a society that trusts machines more than people. In order of importance, people:Want more control over who can access information on themAre reluctant to share informationFear surveillance of their private conversationsWant more control over who can collect information on themWant to be left alone at home
15
http://www.pewinternet.org/2015/05/20/americans-attitudes-about-privacy-security-and-surveillance/pi_15-05-20_privacysecurityattd00/Similarly, individual confidence in information gatherers is low, and decreases depending on who does the asking.People appear to trust their credit card companies, the government and telcos marginally but beyond that, it all mostly goes downhill when it comes to anything online.16
A significant increase in the perception of security was observed with geo-authentication:
Users felt more secureWhen they feel more secure, they are more inclined to potentially be more engaged with the application, presumably because they feel their data is safeGeoauthentication, therefore, could be justification to improve security, but also improve uptake because people trust the application moreAltering Security Perceptions Through the Use of Geo-AuthenticationGongjun Yana, Kenneth Shemroskea & Gary BlackaUniversity of Southern IndianaPublished online: 07 Jan 2015.
A significant increase in the perception of security was observed with geo-authentication:
Users felt more secureWhen they feel more secure, they are more inclined to potentially be more engaged with the application, presumably because they feel their data is safeGeoauthentication, therefore, could be justification to improve security, but also improve uptake because people trust the application more
17
Now, PlaceSpeak does geo-verification better than anyone else. But whats more is that we deliver a rock solid commitment to privacy to everyone. Participants that can trust the proponent with their personal information are more likely to be more open and honest with their answers. Answers that are honest can be relied upon for better policy and decision-making. We authenticate individuals based on their address but like a census, we do not collect specific information on the individual. That sort of data collection is up to the proponent to capture, and to convince the participant to surrender.18
A Broker of TrustReason 1: Privacy & Geo-AuthSimply put, with our Geoauthentication tools, and our commitment to Privacy, we are a broker of trust in engagements. We contend that this is more valuable than having a huge panel of paid survey takers. Because when people get paid for their opinion, they just might be providing an opinion in the hopes that they are selected for another paid panel.19
PlaceSpeak has been developed with certain objectives in mind. Central among these objectives are participant privacy, protection from direct marketing, and the creation of an open and accessible process that accepts all members of a community. In some ways, these goals are at odds with polling and statistical best practices. A consultation is significantly different than a research survey. For example, the more personal information is collected in a survey, the better that survey is from a research perspective. PlaceSpeak, on the other hand, is designed with privacy as a central objective, meaning that demographic data will have to be collected separately for each poll. Another difference between consultation and research polling is that in a consultation, self-selection is a strength: the most engaged and informed members of the community should be more involved; whereas in a research context, any kind of sampling bias invalidates the results at least insofar as they reflect the general population.
20https://www.marketresearchcareers.com/marketresearchprosurvey2007.aspx
Quality of dataQuality of InsightsSpeed of deliveryCostReputationAll these areas are addressed by PlaceSpeaks approach to collecting feedback from participants. And we explore how all these factors are satisfied in the following case study.21
Case Study: The Transportation Vote with PlaceSpeak
Quick backdrop for the out of town folks:
The Province of British Columbia , in connection with the Electoral Agency, held a vote on what was officially termed a Transportation and Transit Plebiscite. Citizens in 23 different municipal areas were asked to consider a 0.5% (1/2 percent) sales tax increase to help pay for transportation infrastructure improvements. After all the ballots were counted, The turnout for registered voters was 48.64 percent. The vote was a non-binding plebiscite proposed by the provincial government and conducted via mail-in ballot.
PlaceSpeak ran a public consultation in parallel with a local news radio station News 1130. Well review the results in the slides that follow.22
At first glance, the site offers a dashboard with some quick statistics about the reach and the participation.
The first number tells us the number of unique views to the page: 5656 people came to see the topic and learn more about the subject matter of the transportation vote. The 5656 number in this context and in comparison with polling activity, would represent the contact rate.This is followed by the number of connected participants which is 900 and represents nearly a cooperation rate of about 16%. Polling is close with numbers just shy of 14%. But where things get interesting is the number of comments at 980. This is because this is not a poll, but a public participation exercise with two-way dialog occurring on the site. But more about that in a moment. 23
The consultation coincided with a media plan launch initiated by News1130 Radio. Most traffic would have come to the page in one of three ways:
Citizens learned about the consultation by listening to the radio, on News 1130AMThey read about the consultation on any one of several popular social media channels including Twitter, Facebook, LinkedIn, or through blog posts on PlaceSpeak or the News1130 websiteExisting users of PlaceSpeak signed up to receive a notification on topics that interest them and were notified via email
24
Reason 2: The PlaceSpeak CommunityEverything a panel can do, PlaceSpeak can do and more. Most polling companies keep a roster of panel participants available for a variety of uses recall the earlier slide about market research versus public opinion? These people are often vigorously solicited for their views on shampoo and voting intention. But PlaceSpeak participants are purely tied to civic duty and community. We maintain that this improves focus and the quality of responses. It also will frequently speak to Category Knowledge as well because citizens know their neighbourhood best!25
Moving forward, when the campaign was initially launched (indicated by the blue line in the center of the graph), PlaceSpeak experienced a proportionate spike in traffic to the topic site. Then, throughout the campaign period, a number of spikes in traffic were witnessed, which corresponded to either media campaigns initiated by News 1130, or some social media buzz that drove visitors to the topic page.26
As the campaign gained momentum, we started to witness some interesting dynamics among the discussion group on the topic. Notably:
There was a spatial variation of responses by municipality week over week this is indicated by the green dots, representing people in the various communities who connected up to the topic to discuss the transportation vote, and how it might affect them, in their neighbourhood;
More importantly, we saw people that were behaving like citizens. Debate on the site was authentic, civil and respectful. What was even more remarkable was the lack of fake personalities that attempted to game the system. We found that when people were asked to provide their address with a full disclaimer and emphasis respecting their privacy people were just more neighbourly to one another. 27
When the results came in, PlaceSpeak was only 3/100ths of a percent away from the official ballot count published by Elections British Columbia. The official vote result was 61.68% against, and PlaceSpeaks participation predicted an overall 61.65% against. But was this just luck? Or was there something bigger happening?28
Elections BC Official ResultElections BC Official ResultCopyright 2015 PlaceSpeak Inc.
Comparing the vote results against a number of popular pollsters, everyone seemed reasonably close. Admittedly, the polls did not reflect the ballot exactly the same, and there was some flexibility or ambiguity that allowed participants to be undecided.
The Not Sure outcomes were not included in the comparison polling; while they could have swung the vote on either side a fair bit in terms of accuracy, nothing was substantial enough to yield a change in the outcome.
In other words, even if we added the 13% undecided to the Yes side, the outcome would have still been a No.29
Copyright 2015 PlaceSpeak Inc.
PlaceSpeaks numbers were modestly different from the official ballot count, with higher support in Vancouver, New Westminster, and the District of North Vancouver. A number of factors might explain these variations including things like sample size, or simple panel bias. And we will talk about Bias in a couple of slides.30
Copyright 2015 PlaceSpeak Inc.
Inevitably, once the sample size gets big enough, things like outliers and errors are diluted and almost unnoticeable. For example, PlaceSpeak did not geospatially segment Tsawwassen First Nation, and PlaceSpeak did not have appreciable participation from some of the smaller villages. Instead, at the time of the vote, PlaceSpeak had higher concentrations of citizens in other areas of the Province; the participation of one community helped to average out the lack of participation from another.31Copyright 2015 PlaceSpeak Inc.Main Reasons for Voting Yes in the BC Transportation Plebiscite Vote
But what about sentiment? Why were people going to vote Yes or No? (Not so ironically) the reasons people gave in the discussion forums were about the same as the reasons provided to pollsters with one key difference: The pollsters asked people why they were voting a certain way, and provided them with a variety of options to choose from. In PlaceSpeak, participants were just asked why, and permitted to discuss their views amongst themselves. 32
N=950 Angus Reid forum members from Feb. 25 to March 5 has a margin of error equivalent to 3.3 per cent, 19 times out of 20 1 2 5 4 3And, while a pollster might claim category knowledge is important, the overall results also suggest that those participating are the subject matter experts. As such, there are probably a finite number of potential answers for any kind of question, and the masses will self-organize, whether knowingly or unknowingly. In this case, the results were remarkably similar: PlaceSpeaks Yes Participants rated (in order of preference):Public Transit Needs ImprovementTraffic congestion will get worseEnvironmental concernsIf we dont do it now it will cost more laterVancouvers future needs33Copyright 2015 PlaceSpeak Inc.Main Reasons for Voting No in the BC Transportation Plebiscite Vote
Conversely, the No side seems equally passionate and sharing the same values as the pollster panels: skepticism, resentment, confusion, doubt all emotions seemed to run quite high in the PlaceSpeak discussion forums.
34
N=950 Angus Reid forum members from Feb. 25 to March 5 has a margin of error equivalent to 3.3 per cent, 19 times out of 20 1 2 4 5 3These results almost mirrored the results of the pollsters. For PlaceSpeak, the order of sentiment was:Translink cannot be trustedI dont want a tax increaseProposed projects dont benefit my areaTheres no benefit to meI dont trust the Mayors
35
So? Whats the Difference?So? Whats the difference?Good question. When it comes to polling:We already have heard that online has overtaken offlineIts more reliable in terms of data collection36
Online is certainly more affordable or can be once the panels are established
37
And an online poll can be completed in a relatively short period of time. A well managed online campaign could hit literally millions of people with the same kind of efficacy as a telephone campaign. Even if an IVR platform (interactive voice response Robocall) was deployed, it would still either cost more or take more time to complete.
38
Big Polling Companies
Value/Software & Self-Serve Providers (Basic Pricing Only)Many companies offering polling services charge a base price, and then augment their services with a variety of add-on modules and upsell the client on things like analytics, access to panels, etc.
39
Bias?But then there is the idea of bias. Many would argue that bias might exist within PlaceSpeak, rendering it fundamentally at odds with getting representative results.40
Reason 3. Acceptable Sampling Bias Some researchers might criticize PlaceSpeak as a research tool that it suffers from a self-selected nature of a user-base. The so-called self selection would be people who are active members of a community, and who are civic-minded.
To them, we simply say thats a sampling bias that we are content to live with. It is certainly better than, say, a pool of paid participants who only volunteer their time when theres the promise of cash at the end of the question period.
Clearly, this case study provides some evidence that the quality of data collection within PlaceSpeak is within acceptable quality standards at least within terms of representativeness and survey-readiness.
And, recall the earlier slide we talked about PRECISION and ACCURACY. In this consultation, the userbase was large enough to provide precision. And, because the users were spread out across the entire voting region, we had exceptional accuracy to 3/100ths of a percent in aggregate.41
Public Participation
Public Opinion Poll
In the context of public participation, you would WANT to have bias. Bias means you have citizens that are engaged with the topic. A poll doesnt generally provide you with background information before it asks you a question it just wants to get a snapshot in time of public position or sentiment.
42
Reason 4: Its Public ParticipationSometimes there is nothing wrong with non-representative polling. A representative sample is only needed to generate results that reflect the general population. But what if a political party or other agency only wants to know about the opinions of the most engaged citizens, for example those who are likely to vote? It is not illegal to administer nonrepresentative samples, and the PlaceSpeak community might prove to be exactly the kind of sample that policymakers are interested in hearing from.43Public participation public opinion poll!Polls represent raw information Neglects processing of complex information Difficult to make informed choiceCitizen engagement is a forum to process complex information Deeper understanding Make a well-founded choice
www.cprn.org/documents/49583_EN.pdfHandbook on Citizen Engagement: Beyond ConsultationPublic participation is not the same as a public opinion poll!
Polls represent raw information that fails to take into consideration the processing of complex information that is necessary to come to what is referred to as a public judgment, or an informed choice. Citizen engagement provides forums for citizens to process complex information so that they can come to a deeper understanding of a situation and thus become capable of making a well-founded choice.
44Then Explain Stakeholder Groups?Stakeholders are often people that have to defend specific positionsLike Translink or the Mayors CouncilPublic Participation runs in parallelInclude citizens in processesRepresent themselvesLike the vote
www.cprn.org/documents/49583_EN.pdfHandbook on Citizen Engagement: Beyond ConsultationHow is this different than negotiations with stakeholder groups?Stakeholder representatives often come to the table with firmly entrenched positions that they are mandated to defend. Citizen engagement, which can be structured as a parallel or complementary process to stakeholder engagement, aims to include citizens in processes, as individuals who represent themselves. Public interest groups sit somewhere between citizens and stakeholders: they take a public interest perspective and may or may not have pre-determined positions that they bring to policy discussions.
45How does citizen engagement fit with the concept of public involvement?
Public involvement is an umbrella term that generally refers to the spectrum of methods with which to consult, engage or involve citizens and stakeholder groups in policy or program development processes. As such, citizen engagement is one of many theories, methods or approaches that fit within the concept of public involvement.
www.cprn.org/documents/49583_EN.pdfHandbook on Citizen Engagement: Beyond ConsultationHow does citizen engagement fit with the concept of public involvement?Public involvement is an umbrella term that generally refers to the spectrum of methods with which to consult, engage or involve citizens and stakeholder groups in policy or program development processes. As such, citizen engagement is one of many theories, methods or approaches that fit within the concept of public involvement.46
So? What? What does this all mean in context of the Transportation Vote? Could the outcomes have been influenced so that a Yes Vote might have resulted? How might have things turned out? Lets get out our trusty 20/20 hindsight glasses and have a look!47Perhaps Reframe The Question:How would you propose paying for improved transportation and transit services?Gas, alcohol, tobacco fuel taxTolls (Commuter tax)LotteryIncreased usage feesAir EmissionsBuild another Casino
Auto LicensingInsurance surchargeSales tax increase (current)Legalize MarijuanaExpand Alcohol SalesOil Pipelines
Lets start with the question that was asked in the first place: The vote asked people if they would approve a half-percent sales tax increase. But from the earlier feedback provided to us in the consultation topic, a lot of people didnt seem to have all the information, or they lacked trust, or they didnt feel like they had a say in the outcome. A yes/no vote isnt a consultation when its the only option thats ever been presented.
Maybe a better question to ask citizens would have been: How would you propose paying for improved transportation and transit services?
Now, this question gets everyone thinking Everything becomes fair game: Sin TaxesUsage taxes or feesEconomic stimulusEtc
48
Unfortunately, the entire plebiscite was an all or nothing proposition: vote for a tax increase to improve transportation. Period. People responded somewhat predictably by voting no. Not because they werent necessarily against the idea of improved transportation, but rather because they werent really a part of the process that arrived at the decision to increase sales tax by a half point.49
900+ Potential Ambassadors?Had the people been a part of THAT process, then the outcomes might have been a little different. The Lower Mainland of British Columbia has a lot of issues facing it. Rather than having a transportation vote, the Province might have had a vote on enabling oil pipelines, or a vote on legalizing and licensing recreational marijuana sales, or changing the structure of automobile insurance in the Province. Any option might have led to a viable solution for could have increased transportation revenues and all options could have had potentially hundreds, if not thousands, of citizen ambassadors advocating on behalf of the Province and the Mayors. This type of Direct Democracy is the most powerful approach because it leverages the power of the masses as the media. The people could have and I would argue they WOULD have owned the entire process had they been empowered to do THAT. 50
We were pleased to find PlaceSpeak allowed us to venture into a deeper and more meaningful understanding of how people feel about this important vote and issues directly related to specific neighbourhoods in the region. We captured elements of differences across the region. As a result, our news reports contained true stories of impact based on geography. Bruce Claggett, Senior Managing Editor
In early 2015, NEWS 1130 partnered with PlaceSpeak to engage our listeners and web/social media followers in their thoughts during the Metro Vancouver transportation plebiscite. We were pleased to find PlaceSpeak allowed us to venture into a deeper and more meaningful understanding of how people feel about this important vote and issues directly related to specific neighbourhoods in the region. We captured elements of differences across the region. As a result, our news reports contained true stories of impact based on geography. -Bruce Claggett, Senior Managing Editor - NEWS 1130 at Rogers Communications51Quick RecapPolling is being replaced by online methodsPeople are increasingly distrustful of all manners of information collectionAuthentication potentially increases the perception securityPlaceSpeak is a broker of trustPotentially yield better results52Quick RecapCommunity plays an important role in Direct Democracy, Engagement, TrustPublic Participation arms the proponent with information to ask the right questions to more likely achieve a desired outcome Public Participation is NOT PollingIts about whats happening now AND the future
53
54
55
About PlaceSpeak and your host56
Contact & Q&A57
58
