73 Opinion Research
We are joined by Adam Probolsky (https://twitter.com/AdamProbolsky), President of Probolsky Research to talk about opinion research and how it can be used effectively by policy makers.<br><br>### Resources and Shoutouts:<br>- https://www.ocregister.com/2020/09/20/adam-probolsky-sees-polls-as-the-bridge-between-the-public-and-policymakers/<br>- https://www.probolskyresearch.com
Top Keywords
- research 0.009
- message 0.007
- data 0.006
- polling 0.006
- water 0.006
- trying 0.005
- message carrier 0.005
- public 0.005
- different 0.004
- whatever 0.004
- carrier 0.004
- election 0.004
Transcript
Hello. I'm Ryan Cook, and this is Civic Tech Chat, a show that looks at the way technology, politics, and policy impacts the world around us. The tools we use, the way services are delivered, and how we talk about and set policy all shape our society. We'll gather around and have a chat about these things together and more. Either that, or maybe I'll rant about a topic for a while. Before we get started, I do wanna let you all know that we've started a Discord for the podcast. There will be a link with an invite down in the episode description. Do feel free to go check that out. It's a small community right now, but hoping to grow it. It's a great way to reach out to me and let me know things that you might want us to cover or to just hang out and talk about civic tech. Anyway, let's go ahead and start the show. Adam, thank you so much for joining us here on Civic Tech Chat. Could you introduce yourself and tell us a bit about what you do?
Thank you, Ryan. Yeah. Adam Prabolski. I'm president of Prabolski Research. We are a Latina and woman owned opinion and market research firm, which means we, do, polling and focus groups and other kinds of original research for everybody from government agencies to corporate, entities to non profits. And, looking forward to chatting.
Adam, what do you what would you say is your personal why? The thing that drives you. Take it out of bed each morning and do what you do.
Is it wrong to say to make money? You know, I I I think that's the the reason I I think I do. Not not not the altruistic one but just, you know, that's what what motivates you to get it done. I look. I'm I'm naturally curious, and I think that's important, in my business. Want to learn. I I like to be the person that learns on a regular basis. So maybe that's a a a more appropriate answer.
Are there any pieces of media, whether a podcast, book, video, or some other such thing like that that you'd recommend to folks to check out?
I'm a big fan of geography, and I think it's, like, really affects everything that we do, and and it's a kind of a lost science. So, there's a book Ken Jennings, has a book on geography, that I think, is is really, really kind of fun, and another one called, Why Geography Matters by a guy whose name I can't pronounce. So I I think I think if if we all had a better sense of geography, I think we'd all, you know, be able to get along a lot better, and have a sense of that history, like Iran. I mean, Iran's been invaded, like, 400 times. So then you get the sense of like, oh, now I know why they're, you know, a little bit defensive. So geography is helpful.
Well, I think we can go ahead and segue into our main topic where we're gonna be talking a bit about things like opinion research and how that feeds into government. As folks might gather from that personal why question I asked, something that I like to do is kinda start with why, as we get, like, kinda going into, into a topic. So, Adam, in your view, like, why should one go about the practice of doing opinion research?
Sure. So, you can fly blind and you can fly by your gut and you can, probably do fairly well with that and quite frankly, most people and organizations do. But if you have the benefit of having a budget, not a massive budget, but a little bit of money to be able to test the message and and how to communicate and and how to kind of get from a to b, you're gonna be just so much better off in being on the mark and being responsive to your constituency, whether, you know, you're a government agency or or a corporation, you're knowing what people really think, how they think, what their behaviors are, and and if you're trying to move the dial to to know what is going to move them. So it it's just logic. The art and science of research, then you're gonna be much better off in having that on your side.
In an interview you did what you did with the Orange County Register, you mentioned that you see polling as a bridge between policymakers and the public. You then go on to describe three things that are foundational to that, metaphorical bridge. Could you talk a bit about that for folks?
Sure. So so this gets into the the specifics of of what we do in general, when it comes to opinion research. Research. What we we tend to do is first test, you know, do you support or oppose? Whatever it is. The building the new sewer plant or or, you know, making the the mayor, you know, an elected position directly elected to the public or, you know, raising sewer rates. What what whatever it is, that you're trying to accomplish. Or in the case of a, you know, business, you know, support or oppose the the the new housing development. And so we tend to test that and see where people are, then we tend to test the message. So and the message is and so let's say you're trying to build new houses. Well, you know, you might have opposition to the homes, but you might have support for the million dollars that are gonna be put towards transportation, improvements or the the $2,000,000 that can be put towards schools, or or whatever other resources that are out there. So you're gonna test those different messages to see, okay, what brings people on board to be supportive of whatever it is you're trying to accomplish? And then so that's the message. Then there's the mode. How do they communicate? So we'll probably talk about it more later, but in polling, right, we are now in a multimode scenario where we talk to people on the phone and on email and on text and and in the mail and all these different environments online. So, we need to figure out where people are spending their time. Are are they all on Nextdoor or are they all on YouTube or are they all still having landlines, you know. And and making sure we're communicating with them how they like to communicate and getting the messages at the places where they spend time. And then the third part, that's the, like, the mode. And then their third part is the the message carrier. Who is it that's gonna be effective? Is it is it the mayor that's gonna be effective? Is it the CEO that's gonna be effective as a message carrier? Or is it the public works, you know, guy who or gal who have a helmet, you know, a a white helmet, and and are are you know, look like they're doing something with their hands, you know, or is the scientist with the pocket protector, or is it the spokesperson in a suit? Who is the best message carrier to kind of communicate what what what it is you're trying to communicate? Who's respected? And that could be different, by the way, for different groups. So one of the things that polling tells you is, yeah, overall, you have support or overall, they like the message of of, you know, more money for schools. But it turns out that seniors care more about water quality and, you know, younger people care more about traffic. And and then and seniors, you know, are more much more respect the general manager of the agency versus young people who, you know, respect the the local influencer or whatever, you know, thing we're trying to accomplish. So, really, it's it's it's part of the value of the research is to really be able to segment your mode and your message and your message carrier to really focus in on who it is you're trying to to move or who it is you're trying to educate and inform.
Adam, I think something I've been hearing in there is that you might actually do some research just to figure out who that best carrier is to kinda and try to match that as concerns. Am am I hearing that correctly there? Yeah. For sure. So so,
on a, you know, water conservation message, who is it that you're gonna trust? Is it is it, the the director of the state water agency, or is it the, you know, the the the climate action scientist, or is it the the environmentalist, or or is it the the mayor? You know? And and and and try to figure out or the legislature or the governor, try to figure out who it is that people are gonna be responsive to. And oftentimes, you know, someone or some entity rises to the top and we, we kinda recommend that be the the the main focus of who the message carrier should be. But like I said, sometimes different groups are much more responsive to different types of message carriers, and you you have the luxury oftentimes, especially in a digital world, of segmenting who you're talking to in those different environments, or in direct mail even, and and using those particular message carriers that are most effective.
Something I'm I'm kinda curious about. I actually read this this is definitely not question from the sheet, but, you know, folks have seen kind of, effectively, like, polling come out about, like, level of trust in government, which I think would maybe relate to something like as you're say if you're looking at should I have a person in an authoritative position of government be the messenger for this? Is that something you've seen bear out in in your experience in recent times or or not so much?
The truth is about six years ago, we saw a precipitous drop in trust in government and all institutions in this country, or just all institutions in general. There was just a collapse of trust, in in any kind of formal entity. And we have started to, in the last year and a half or so, kind of come back a little bit from that. But but, yeah, that's it's a big problem because it's not just, you know, the congress. Right? We used to have the scenario where, you know, I'm kinda stacking my hands here, right, where where local government is on top and legislature, whoever whatever state it is in the middle, and, you know, the US congress is is the bottom. Right? Where you but but this place where we held up high our local agencies, our regional agencies, our counties, our our our cities, and and we felt like they were we were close to them and they had our best at heart. And we saw that really just bottom out altogether, and the same thing went for even nonprofits and and and, you know, everyone from the CIA to the to the the the the local parks district. So, I mean, it became a real problem to find the message carrier. I think we're seeing that come back a little bit, and we do see pockets of, places like for for specific and I talk a lot about water because we do a lot with water, but water agencies do still have a lot of credibility, and certain local agencies still have credibility. And there are outliers where where the the city or the county has kind of, built a lot of great trust over the last couple years, and people do have faith in them. But in general, it is a challenge because institutions across the board are way lower than where they were before.
Before. I think you started to allude to this a bit, from the original question, but, I imagine it can be kinda difficult these days to get a representative sample as you're trying to figure out what a community is thinking on an issue. And you mentioned things like there's different mediums, communication, and technology has changed as a bit. Like, for example, I'm landline use has gone down quite a bit, and I've heard that cited, in commentary about about polling. Is, I guess, is that particular example something that's, particularly challenging and what other challenges have you run into with those communication mediums?
So I hate to be the one that's contrary, but, the narrative that is harder to pull today than it was at any other time is so off base, and and it's really easy for me to explain it. Twenty years ago, we had to call you at dinner time. It was the only way to get a representative poll and convince you to, you know, step away from your your biscuits or whatever it was and talk to us for twenty minutes. Now I get to call you on your mobile. I get to send you an email. I get to send you a text. I can send you a piece of mail. I can send you, you know, with a QR code. I can talk to you while you're walking the dog because you've got AirPods in. I can talk to you while you're driving to the office because you've you're on speaker. There is so much more opportunity, for me to for our interviewers to talk to you or let you respond online at 02:00 in the morning or 02:00 in the afternoon. So it has only gotten dramatically easier actually to do survey research, to get responses, to be representative. There's all kinds all kinds of new ways. There's panel, people who sign up for panels to to respond to surveys. I can recruit people by social media, really targeted, not just like, hey. Do you wanna take a survey and hope they live in, you know, in Dallas in Dallas? I can actually send, I can it's called a custom audience. I can target only people in Dallas, by uploading a list of consumers to Facebook and and and have ads going directly to them on Facebook and and Instagram only to the people that live in those communities. So and and, obviously, you can call and and, it is infinitely easier to to get the job done. It's a narrative that I think somehow, maybe some pollsters started because they wanna charge more money. But it is actually less expensive today than it was before.
I I think so what I'm hearing hearing there is that, like, there's more opportunities for contact and that maybe even the medium itself becomes kinda data that you have. It it itself kinda communicates something. Am I am I kinda hearing that correctly?
So data is a a a big part of of the business beyond the answers to the questions because we have underlying data. So, let's say we're dealing with something that's on a policy level or an election level, you know, for for a city or county or even statewide ballot measure. Then you're using typically the voter file, which has, you know, most states has a very rich set of information. Your age, your gender, your your where you were born, the the your language preference, your party, all those different things. Incidentally, in Hawaii, they they don't tell me how old you are. They they stop kinda looking at that in, like, 1992, which is kind of annoying. But in most places, they have a lot of data. You know, how often you vote, when you vote, when you return your ballot, if you're voting by by mail. So that is a big part of it because I can use that as a starting point before I even ask questions. We can do the analytics to to get some of the initial understanding and answers. And then, on the, on the consumer side, if we're not dealing with something that's election related, but still most likely public policy, which is a lot of what we do, but it could be market research too. We are, for better or for worse, a data driven society where, you know, I I don't know how much money you have in your bank account, Ryan, but I know you bank with Wells Fargo. And I know there's between 50,000 and $250,000 in there. And I know that that you, you have six credit cards. And and, you know, I'm I'm kind of pulling these numbers out of out of thin air. But but the truth is if I looked you up in in in data, I would have that information. It it you know, Experian and all these other companies collect that, and we have access to it. So so, yeah, we have our starting point today is much, much better than it was years ago because we have you know, I know what's the magazine subscriptions you have. So I can I can kind of presuppose, you know, that that you have a cat and you own a gun and you, you you you know, go to Disney, on a regular basis because of your consumer habits that are fully available? And we can all lament, like, how outrageous that is, but the data's out there. So we're gonna use it, and and it's just the reality of the consumer society we live in today.
And the, the listeners can have fun trying to figure out which of the random examples actually might have been in this this case. Exactly. So so you bring up data, and I think something that I would imagine happens is you're you're getting data from a bunch of different sources of varying types. It can kinda go into a research engagement. Bit about the process of distilling all of that information down into something that leads to, you know, maybe something actionable or something you can draw a conclusion from?
Sure. So we start with the the first, start starting point is universe. Who are we talking to? Are we talking to voters if it's something lecture related? Are we talking to consumers if it's something not specifically elect or election related, that that makes it not allowed? You you can't use the voter file unless it's lecture related. And then from there, we sit down with the client. And if it's a company that's trying to sell widgets or or it's a a city that's trying to, you know, check on their customer service and what people think about, you know, from a best, management practices, best practices, how are we gonna improve our services and be more responsive to the public? Or, you know, we've got a finite, discretionary budget. You know, we we spend $50,000,000 a year as a city, and and literally, like, 5,000,000 of that isn't programmed. Where should we put that to be responsive to the public? So we then sit down with the client, and these days, it's a lot of time on Zoom, but we we love being in person. And we say, okay. What what do we wanna accomplish? What do we wanna learn? And and oftentimes, we're using questions that are kind of similar to other public agencies because they wanted to have comparison data. But most of the time, it's kind of unique things that we're asking about that community. And then, we we, have the questionnaire that's that's approved by the client. We do the the field work, meaning we call people, we email them, we text them, we send them, you know, mail, whatever it takes in that particular community to make sure we can get a representative sample. And that is so critical here. Making sure we have the right number of, you know, people in each age group, in in each gender, ethnicity, all those things to make sure we're really matching that community when we're done. And then, in the end, yeah, that report is actionable. Our goal is our our biggest fear is kinda delivering some sort of mind numbing set of numbers that no one's gonna do anything with. Our goal is to give these, like, very quick actionable moments of, you know, here's what they think and here's what you can do to make a change. You know, here's what they think and here's why you they think you're doing a great job at it. And so, it's it's our job from a data visualization standpoint to make sure that everything we provide is is actionable, on the part of the the the agency or the client we're working with.
As you're working with those agencies and clients, have you ever found yourself in a position where you've had to kinda advocate like, hey, like, no, this data is useful to what you're trying to do. And if so, like, how have you gone about trying to tell that story? Because I imagine it can be difficult at times.
Yeah. And it it happens, obviously, it's much more fun when you're working with a client who trust the science of of research and and the math and appreciate it and and really kind of want to, use the data to to the benefit of the community or or their constituency. And and, yes, there are definitely moments where someone says, I don't believe you or or, you know, I didn't get a call so it can't be real, or or, it just doesn't fit their narrative, their their their idea of how things are supposed to go or what people are supposed to think in their minds, so they're gonna discount it. And, there's there's not a lot that can be done other than point to the body of work out there of opinion research that is is, you know, 99.9% accurate. And you can point out some of the really negative and and and, things that that show that they're they're not accurate. But in the vast majority of of of, professionals who do what I do, the results we deliver are accurate at that moment. Now, obviously, factors change. So if you say, oh, well, the election changed and, you know, six weeks later, something different happened. Well, we can explain that away. But if someone generally does not want to believe in the data, there's not a lot we can do. You know, it's, if you have an agenda that is counter to what the results are, I'm probably not gonna convince you. And, and that's probably pretty sad because, you can, you can have a different viewpoint. I I love the phrase, I disagree. I really despise the phrase, you know, expletive, expletive, expletive. You know, you know, I hate you. You know, it it's just not productive. And I'm a I'm a policy guy who who loves data, and and, you know, we generally you know, we don't get it wrong. I mean, we we we have an accuracy rate of a 100, when it comes to predicting outcomes. So I I I can only point to the data and then that's about as far as I can go.
As as you think about on your work, putting together the sort of data, are there are there any any anecdotes top of mind for, like, oh, like, we did this research and it ultimately, like, led to this, you know, public policy proposal or or change, that you'd be willing to share?
Sure. I mean, all the time. I mean, we did we do, the poll in a little town in New Jersey that they asked the public whether they they wanted to pay for a community center. I mean, it it's I know it sounds like, you know, kinda silly, but boom, it tells you right there, they're not willing to spend the money, so the city isn't the town isn't gonna spend the money. Or right now, we're doing a lot of work on conservation in the Western US, trying to understand you you know, we've been in the Western US, we've been, you know, under kind of drought conditions, on and off for, you know, for decades. And and so there's this constant drone of use less. You know, you do more with less. You know, conserve or be efficient. And so we're at this point where we've gotta make incremental improvements, and how do we do that? So we've gotta ask the questions. Do you think you can do more? You know, what is it? Would you would you rather, you know, have a a rebate on efficient applying water efficient appliances, or would you rather have someone, you know, come to your home and give you tips on how to fix things? I'll tell you right now, they do not want anybody coming to their home. You know, they do not want, you know, we're from the government. We're here to help you save water. They want, you know they'll install whatever appliance you tell them to install. Just don't show up. And so our job is to is to figure that out. And, yeah, there's there's constant examples of, you know, figuring out what that nuance to the message is to get people to be a little bit more, conscious of their conservation, and water efficiency. And the same thing goes for for, let's say, the kind of typical customer service or issue or or budget prioritization, where, you know, a small town or or even a big city where you have the public saying that they're they're really upset about sidewalks. You know, an older community where you've got, trees that are encroaching on sidewalks and and, you know, pushing them up. You you've gotta have that balance between preserving trees and and and, and keeping sidewalks safe. And so sometimes the polling says sidewalks are the priority. It's okay to get rid of the trees. And, you know, it's I know that's, like, sounds like super detailed in the weeds, but that's what public policy makers have to deal with on a regular basis, and that's what staff has to deal with on a regular basis. You know, how do we make these hard decisions? And and research will tell you oftentimes, is it okay to have crappy sidewalks, or is it okay to get rid of the trees? And which one you're gonna do first?
Something I'm kinda curious about since you mentioned the, like, conservation and kinda like the Western Region specifically is, I've heard that, like, conservation is probably something, particularly in that region, that's a concern that kinda crosses political ideologies, perhaps because it's something that's, like, kinda like more real and and in your face in in that region. Is that something you've kind of seen bear out? Like, is there general consensus? May maybe not like, hey. Someone should show up to your house as as you mentioned, but, like, that some community action is probably needed, to address those concerns.
I I think one of the challenges is, everything's become partisan, and everything's become so divided. And so, yes, in in a place like California or in a place like Nevada where, you you know, from a a a water resource standpoint, we all know that we're we're we're short on water and we've gotta make a change. Yes, I think there's kind of general consensus. But as soon as you start talking about the solutions or long term solutions, you have big divergence in in in support. So for instance, you've got this this kind of conservative, you know, kind of republican understanding of we need more surface storage. We need to to store more water. And then you've got, you you know, maybe a more progressive, or or democratic perspective of we need to, you know, maybe limit growth or or or or, you know, change change the way we we, we deliver water. And and there's this, like, you know, that you're gonna start very quickly becoming partisan, once you get beyond, yeah, we need to, you know, we need to have a better water resource, you know, you know, goal.
So I I think what I'm hearing there is effectively, like, same goal, but different different paths to get there.
Yeah. Yeah. And I I think that's just the case for almost every public policy, issue these days. We're we're starting to find that as and maybe to some degree as partisan or as just kind of it's just diverging paths, and and and part of it also is is the the, democratization of information, and and the ease of which we can we can spread disinformation. You know, your your your city has a $50,000,000 budget. You know, 22,000,000 goes toward cops. You know, 18,000,000 goes towards towards the firefighters, or or, you know, whatever. There's a, you know, $12,000,000 water department. I mean, the the money is, like, specifically programmed. It's not like, it's not like, you know, there's money flying out the door to to, you know, to to to to people going on vacation to Hawaii. So so really but the the narrative sometime on Nextdoor could be very easily, you know, o m g. You know, they just bought, you know, a new, you know, all new police cars, and they didn't need them. Or or I can't believe they just spent $50,000 on a new website. You know, those are the those are always the things that people get so upset about. The ones that they all kind of understand because, like, they can build a website or WordPress site for $12. So it's like, how could you possibly spend $50,000 on that? Or or the, the the rebranding. You know? We don't need rebranding. We don't need to attract, you know, tourism and, you know, and and you can make that direct correlation between spending a $100,000 on rebranding and and, you you know, the the millions of dollars in revenues you get from from tourism or from from from business, coming to town, but but it's really easy to throw throw rocks at at something like that. And and so that narrative quickly can get out of hand. And which, by the way, is one of one of the reasons why why the the, the research is so helpful because you have this moment where where you have these 16 people showing up and being angry or 50 people on next door saying mean things. And then you can show from a public opinion standpoint, broadly, people don't care or broadly people are supportive because they're they're just not as engaged as those 50 people. They just generally say, yeah. City's doing a good job or, yeah, it's smart to spend the money in that way. And you can kind of counter that that negative messaging that's
out there. As, we continue to talk about and and mention polling. I I think, a lot of folks, when I say that word, maybe start to picture those kinda like horse race polls that, you know, there's a lot of stuff on the news about as you get closer to an election. It seems to be, like, the the the hot item in those stories. How does the work that goes into those differ from the sort of research that we've been talking about so far?
I mean, so first, foundationally, it's a voter file, which is tend what we tend to use. And, and that, you know, gives us that perspective of of all the background information, not the consumer stuff, but the the party and the age and the gender and all that that kind of similarly is on consumer, but more data on the the about about politics on your on your election file. So foundationally, that's the big difference. And then, obviously, our goal is, to understand who's gonna win. You know? It's it's not just what do you think and how do you feel and what messages sell. It's it's oftentimes, you know, in a political sense, it's it is the horse race. Does measure, you know, a pass or fail? And then what are the motivational messages that'll move you towards it? So it's not that different than, you know, about measures, not that different than a candidate, than a than a they're basically the same thing. We're just trying to figure out where we are today, what messages move people, pro and con, and then and then what's gonna happen in the end and predict the outcome, and and drive the the the outreach message, the campaign itself. Whereas policy wise, you know, we're, we're not necessarily trying to always move the dial, although sometimes we are. And we're always super conscious, depending on the laws of the state, you know, almost in every case, government can't advocate for, the passage or failure of of a ballot measure or or a candidate, of course. But they can certainly be in the business most of the time of, educating the public on the the pros and cons of, let's say, a ballot measure, passing or failing. You know, you can say, look, if this ballot measure passes, we get to have, you know, pretty parks and and and green grass and and and a playground for everyone. If it fails, we're gonna put chain link fences around the parks. They're gonna go brown and nobody gets to play in a park. Now, that's kind of a dramatic, messaging there. But that's the kind of thing that could potentially be done, by a local jurisdiction or even a state agency, versus, versus the actually advocating where in election polling, it's it's it's a campaign whose specific agenda is to pass or fail a measure or a candidate.
And and they're much more aggressive in their words and much more vote yes, vote no, whereas government can't do that. Yeah. And as we're talking about those campaigns that kinda, you know, they have an agenda, they're kinda being aggressive by a thing they want they want you to think. A, a technique that, often is used by those organizations is a thing called push pulling, where effectively maybe is more of a tool of persuasion than about, like, trying to figure out where people are. Is that a practice that you think is effective or, like, what I guess, like, what's your hot take on? Yeah. Let let me let me let me that that's a very sensitive subject for pollsters.
No pollster you've ever heard of, seen, whatever, you you know, engage with has ever done a push pull. Pull. A a a a push pull and it's a crappy set of words because it doesn't it's it's very incongruent to the research. What what you'd consider a push pull is, someone, a campaign calling 10,000 or a 100,000 people or some massive number of people saying, hey. Did you know that that Smith, got a DUI and and ran over three people and and, you know, served, you know, three years of probation? No? Okay. Thanks. Goodbye. You know, that's a push pull. And and that's and that's just a a a crack of shit. Right? It it it's it's it's it's not, you know, in any way a set of research. Research will what we do, we absolutely ask a question that is aimed to push people in one direction and see if they are responsive to that message and see if they will gravitate towards that. But we are asking 300 or 400 or 800 people or 1,200 people, whatever it is, a question in order to understand the out the the research. And we're never gonna say something that's totally outright a lie. Right? We're never gonna call someone a child molester when they're not. We may ask a question about the DUI and say, if you knew Smith had a DUI and served three years of probation, would that make you more likely or less likely to vote for them in the context of, you know, 20 or 30 or 50 other questions? But but it's really in the in the interest of research, not in the interest of of educating the public or or well, I mean, not even educating. The the in the interest of, you know, messing with with people's minds and and, putting a message out there. That is never the aim of research.
And I I think you might have been getting into the space with, kind of digging into talking about this being a bit of, like, a sensitive term and topic for folks in the profession. Yeah. Is that something y'all struggle with is, like, that kinda con, conflation between the two things? Like, that first thing you described where it's, like, clearly, oh, I'm trying to make cause you to think this thing. It's not research, versus, you know, your other methods that that introduce that research rigor is more of a curiosity based thing than a persuasion thing.
Sure. So if you if you look at Nextdoor, when any pollster, any researcher is doing research in any particular community, the people that don't like the questions or upset at city hall or the, you know, county hall administration, are going to typically in in those next door conversations say, oh, this was a push poll because they were only asking questions about blah blah blah. And so sure, it it it's it's definitely something that comes up. And then it's our job as researchers when we make a presentation to the the policy makers, the council, or staff, or whatever it is to say, you know, to counter that, make clear, you know, this was research. We did, wanna test messages, and we did. But in no way were we trying to influence the outcome, broadly. We might have been trying to influence, people who are responding to the survey to see how they would respond to messages. So for instance, for a tax measure, we might say if you knew that, you know, everybody's, you know, would get puppies and kittens and and you, you know, you'll have rainbow Skittles for everybody, you know, if you passed. And and if it failed, you you know, we would we would, have an earthquake and and, you know, the the demons would rise from the from the earth. Would it be more or less likely? You know what I mean? We we might ask questions that are are really, charged. But, again, the goal is to understand what people's response would be, and and how they would be, how would they be moved by those messages. Sometimes, things that are are things that city hall might talk about, positive stuff, and also sometimes what the the naysayers, what we know the taxpayer group might say. We wanna make sure we ask those questions to understand the real impact on the outcome because we ask the ballot question again at the end. And we we you know, to see how those push and pull, you know, the neighbor to neighbor conversations would affect the the outcome of the of the election if once we test those things.
I think something I'm hearing there is that at times, folks, even public officials might receive, like, the results of your research through their own lens and kinda with their own, like, internal biases there already. Is that a struggle you've had with, like, folks you've worked with? If so, like, how do you go about addressing that in the conversation?
For sure. It's a it's a big part of what we what we deal with. And, what we do is try to set up very early on the process that this is research. It's statistically valid. It's representative, and this is what your public thinks. And and it may be counter to what you think. It may be counter to what what you've heard out there. You know, it's pretty hard, I think, for policymakers because they're in the supermarket. They're, you know, in the public. They they go to events all the time, ribbon cuttings or whatever it is, and they hear from their constituents. And oftentimes, it's a little bit of an echo chamber, and and they really, you know, just hear from the people that wanna tell them what they wanna hear or people complaining. And so, yeah, it's a bit of a challenge for us just to make it clear that this is math. It's a little bit of art, art, but mostly mostly the science of re research, and and these are the real numbers that represent your constituency. And and even if you don't think it quite aligns with what you're familiar with or what you've heard, the reality is it is real. And and usually, there's some validating data throughout the poll, throughout the survey or focus groups or whatever we're doing that they can say, oh, yeah. I believe that. So it's harder for them to say, oh, I don't believe the other one. So hopefully, they they find something that they can latch on to even if they don't necessarily want to believe the data.
So, Adam, it's not, you know, every day on the podcast that I get to have someone on with, you know, expertise in polling. And, so I there's a there's a kind of a question that I almost feel forced to to to try to ask out of curiosity, and that's that folks might be aware that there's been this kind of commentary that happens with each election cycle in recent times that, oh, the polls have gotten it wrong in some way and, like, trying to predict the result. Like, what's your perspective or dare I say, hot take on that narrative?
So at the the risk of of, sounding, you know, self serving, we have a 100% accuracy rate at predicting outcomes. So I I and and I'm I'm an advocate for our industry. It's not, you you know, you don't have to hire me. You know, you you can hire any researcher that that's good at what they do. And so I don't, I don't really know, necessarily what, I I don't know what the problem is that that those folks are having, getting the numbers right, some of those people. But, I'll tell you the the challenges that I think some people face when it comes to polling. Nationally, it's kind of hard because you do have 50 unique states, so presidential is a little bit more of a challenge. I don't do presidential polling. So maybe that that makes it easier for me. But, but if you're if you're not engaging, if if your pollster is stuck in 1996, then there's a good chance that they're not going to, get the numbers right. So if you're not embracing the multimode environment, if you're not meeting voters or or people who are responding to your surveys where they are, which is largely online today, then you're gonna have a a challenge on your hands in getting good data. The other thing that I think is more important than anything else is the poll of polls, which is this this thing that we we we desperately crave. We, as as Americans or people in general, you know, I wanna see the aggregate of all the data and tell me the answer. And there's nothing accurate about that because the methodology of poll a is different than b is different than c. And all of a sudden, you put those numbers together and you just have a massive pile of garbage. So, when you when you see a poll of polls, some aggregated dataset, it is it is totally useless. But we like it because it's fun to look at and it makes us feel like we're reviewing doing something sophisticated. We're taking some sort of numbers and putting them together and being sophisticated in our approach. So I think that's the biggest problem I've seen. But I'll give you an example of of a thing that we don't do that you think everybody else does. If I was calling you and asking you a question about the upcoming, you know, midterm election in November, I would never ask you if you were planning on voting. Because the truth is, I already know whether you're gonna vote. I have a very clear picture of your voting history, and I am not gonna ask you that question because you lie. Why do you lie? Because the American experience is that we are a democracy and that we're we're, you know, we're not really, but but we're we're a democracy and that we we we should all vote. Right? My social contract is that I'm gonna vote because it's the right thing to do. So the actual turnout is gonna be, whatever, 42% and in the primary and and and, you know, I'm gonna get 77% of people say they're voting yes. In what world is that useful for me? So we really need to look at the previous data and the the analytics to to inform who it is you're even gonna ask the questions to. And and that's, like, a big factor. So, I think there's a ton of great researchers out there, ton of great pollsters. And then there's some people that are just putting out crap numbers, and you've gotta be a little discerning. And and if someone is is is, you know, you can be a partisan if you want, but you've also gotta be, you know, accurate in your data. And and if you're you're you're not realistic in what you're presenting, you're gonna have a problem in in it. And and sometimes people just do that sort of thing for press. But it's it's, their credibility just gets killed the minute they they they're really, really wrong.
Adam, thank you so much for for taking the time out of your day to join us here on Civic Tech Chat. I have no doubt that folks have gotten to learn, a bit more about European research and how it might, you know, serve a purpose in communities around them.
Thank you very much for having me.
You can follow us on Twitter using the handle at civic tech chat, visit us on the web at civictech.chat, or subscribe
