Berkeley, California - One of the largest experiments ever conducted on people was carried out during the 2012 presidential campaign. Barack Obama’s canvassing teams tested different persuasion scripts on millions of potential voters and donors. Later, when they canvassed again, they used only the proven winners to convince people to vote for Obama.
Most persuasion experiments are now being conducted online by advertisers and marketers to devise the best way to influence buying behavior. But a growing number are being run by political campaigns, public interest groups, and academics to better understand how citizens can be persuaded on issues of public importance.
Persuasion experiments are a fairly new component of elections and public opinion surveys, says Jasjeet Sekhon, professor of political science and statistics. He should know. He and his graduate students are pioneering powerful, new statistical approaches and computer algorithms to improve and streamline surveys.
They have devised strategies to greatly reduce the sample size needed in the experiments without compromising reliability. This in turn cuts down the time and cost of the field research, bringing it within reach of organizations on tight budgets. The refinements have been particularly helpful to public health campaigns and clinical research.
Sekhon is working with public health officials to boost acceptance of vaccines. He also develops and evaluates
persuasion tactics to help move the dial on public acceptance of climate change.
“Most of the work we do is with non-partisan, non-profit organizations, and companies” Sekhon says. “My grad student Josh Kalla has worked some with partisan political campaigns in the past, so he has a lot of experience in this area.”
Sekhon’s innovative research has been recognized with a Signatures Innovation Fellows award, a new UC Berkeley program for visionary faculty entrepreneurs, intended to bring new technologies to the market.
“Right now, it takes a lot of human effort to implement our experimental design and our statistical and computational methods,” he says. “Our goal is to develop software to make this easy for organizations big and small to implement. The current focus is on persuasion experiments, but the methods and design have all sorts of applications, such as online marketing experiments.
The reason persuasion campaigns can work, he says, is that people often have conflicting ideas.
“They may consciously believe in equal rights for gay people, but also have an emotional prejudice. Or they may believe in the benefits of modern medicine and the public good, but fear vaccines or not trust doctors and the government.
“In persuasion campaigns you have a dialogue, and try to get people to link the policy issue or medical treatment to their pre-existing belief. In the case of vaccines, the goal would be to have people who believe in the public good see that not vaccinating their own child puts other children at risk.”
Late last year, a paper in the journal Science reported that attitudes toward same-sex marriage could be significantly changed just from short conversations with gay canvassers. Kalla and then-fellow grad student David Broockman thought the study’s numbers didn’t add up. After they flagged the study, other inconsistencies emerged, and in May, Science magazine retracted the scientific paper. The flap over the bogus study made The New York Times and other outlets around the country.
The gay marriage persuasion experiment may have looked too simple to believe, but Sekhon has devised sophisticated statistical tools that allow researchers to better design and analyze these experiments. In one scientific paper, “How to Measure Persuasion 98% Cheaper and 90% Faster,” Sekhon, Kalla and Broockman seem to have turned conventional polling wisdom on its head. The long-accepted way to measure the presence of a certain trait in a population — whether it’s the response of patients to an experimental drug or the effectiveness of different persuasion scripts — relies on large numbers of randomly selected targets.
But Sekhon’s team devised algorithms and new statistical methodology that allow experimenters to gain reliable assessments on people’s attitudes by tapping a fairly small number of respondents of the larger population, and only those who had previously been identified by a previous survey. In other words, using these new strategies, a small, non-random sample could yield precise results that would have otherwise required a much larger, randomly selected sample. As their paper’s title claims, this can provide a much cheaper and faster way to get to the desired end point.
“These kind of face-to-face persuasion experiments have only been carried out in the past few years,” Sekhon says. “We are throwing a lot of statistical algorithms at the problem. They are the key for getting these efficiency gains.”
By making persuasion experiments much easier to carry out, these potent tools will no longer be limited to organizations with deep pockets. Strike another blow for democracy.
