Problems and Purpose

Deliberative Polling® is a unique form of political consultation that combines techniques of public opinion research and public deliberation to construct hypothetical representations of what public opinion on a particular issue might look like if citizens were given a chance to become more informed. As a polling method, the Deliberative Poll seeks to account for the preferences and opinions of citizens both before and after they have had an opportunity to arrive at considered judgements based on information and exposure to the views of fellow citizens.[1] The technique was first developed by Professor James Fishkin of Stanford University in 1988, and has been adopted by local and regional governments across the world – including Canada, the United States, Greece, Italy, Bulgaria, Hungary, the United Kingdom, Brazil and China.[2] In these diverse settings, it has been used as a more accurate means of polling the public, as a method of developing citizen preferences on difficult issues, as a mode of selecting candidates to run in competitive elections, and as a policy tool to direct political officials in public planning efforts.

A method related to deliberative polling are Citizen Dialogues.

Fishkin initially designed the Deliberative Poll in response to perceived defects of modern public opinion polling methods.[3] Fishkin’s main problem was with high levels of citizen-ignorance on many public issues that required substantive background knowledge in order to take an informed position. In these scenarios, citizens have an incentive for ‘rational ignorance’ – that is, they have an incentive to remain ignorant about an issue because the costs (such as time, energy, etc.) of educating themselves outweigh the benefits of doing so. When the incentives for rational ignorance in the general population are high, traditional sample surveys of mass public opinion will often turn up ‘non-attitudes’ or so-called ‘phantom opinions’ on many public questions. This means that ordinary polling techniques will offer, at best, a snapshot of public opinion under conditions where the public has little information, attention, or interest in a given issue.[4]

Given the problems for accurate opinion sampling under conditions of rational ignorance, Deliberative Polling is meant to offer a representation of what the public would think about a specific issue under ‘ideal conditions’ – that is, under conditions in which citizens have access to fair and balanced background information, are able to account for the different opinions of fellow citizens, and can interact with expert panels of policy-makers to understand the tradeoffs of different policy positions.[5]

Origins and Development

Fishkin first proposed the idea of a Deliberative Poll in an article published in Atlantic Monthly in August 1988.[6] The article was later developed into a book in 1991 entitled ‘Democracy and Deliberation: New Directions for Democratic Reform'.[7] In both works, Fishkin consistently draws upon two primary inspirations in setting out a viable model for a Deliberative Poll: notions of direct democracy exhibited most prominently in the classical Athenian practice of government; and, the foundational work of George Gallup, whose pioneering work on modern opinion polling still forms the basis of most modern polling practices.

The first parallel Fishkin draws, is between his Deliberative Polling proposal and the democratic practices of ancient Athens. According to Fishkin, the Deliberative Poll directly appropriated two distinct features of the Athenian model of democracy: first, citizens were selected to participate in Athenian institutions through a lottery, which Deliberative Polling replicates by relying on the method of random selection; and, second, Athenian citizens were typically remunerated for participating, which is a principle that Deliberative Polls also utilize in order to ensure that a more representative microcosm of the public is constructed by encouraging participant heterogeneity.[8]

Fishkin combines the Athenian model for democratic participation with the public opinion polling methods of Gallup. According to Fishkin, Gallup had originally developed public opinion polling with a view to restoring the intimate face-to-face model of democracy of the New England town meeting to the larger state. Figuratively, Gallup had argued that citizens should regularly gather together, make and hear arguments from political leaders to the public, and vote for or against policy proposals directly. In this context, the public opinion poll was seen as completing the political conversation by carrying the public’s views back to political leaders.[9] The problem, however, is that often times respondents to opinion polls are unable to give well thought out answers or informed opinions about an issue – leading to polls becoming an instrument of political elites rather than an empowering democratic procedure. Deliberative Polling, ideally, is supposed to narrow the gap between well-informed and actual public opinion.[10]

The first ever Deliberative Polling experiment was held on April 15, 1994 in Manchester, England. A randomly selected sample of 301 British citizens participated in the event, and over the next two days considered and discussed a range of policy proposals to deal with crime. During the event, participants met, deliberated in small group-sessions and then put questions to and heard answers from a panel of experts and representatives of the Conservative, Labour, and Liberal Democratic parties. At the end of deliberation day, participants were then asked to once more provide their opinions, which were recorded and later broadcast in a two-hour television special shown on Britain’s ‘Channel 4’. The Poll was viewed as largely successful, with before and after opinion poll results showing that almost half of participants change their initial positions: nearly 40 percent adopted a more moderate position, and some 10 to 15 percent switched sides completely.[11]

As of 2008, there have been more than 50 deliberative polls around the world. Eleven face-to-face polls have been national – with five of those in Britain (on crime, Britain’s role in Europe, the future of the monarchy, the 1997 general election, and reform of the National Health Service). Other polls have been conducted in Denmark (on the question of adopting the Euro), in Bulgaria (on crime), Australia (one on Australia becoming a republic, and another on Aboriginal reconciliation), and the United States (one on foreign policy, the American family, and economic policy, and another on foreign policy in the lead up to the second Iraq war). Thirty-nine of the polls have also occurred at the regional and local level in both the United States and China (notably, in 2005 on participatory budgeting in Wenling City and another similar poll in the same city in 2008).[12] While most Deliberative Polling experiments have been face-to-face, two successful polls have also been conducted on-line – although they still took place in ‘real time’ and used microphones so that participants could hear their fellow deliberators.[13]

Fishkin has registered both ‘Deliberative Poll’ and ‘Deliberative Polling' as trademarks in order to supervise the quality of polling experiments, as well as raise funds for further research and application of the method through the Center for Deliberative Democracy at Stanford University. Fishkin has also served in an advisory role on all polls conducted to date.[14]

Participant Recruitment and Selection

Citizens are usually invited to participate in a Deliberative Poll through a process of random selection. The random selection process can occur in either one of two ways: through random digit dialing; or, by sending out ‘warm-up’ letters to a random sample of phone listings and followed up with phone calls. In order to mitigate problems of selection bias, several calls are made to each phone number to ensure that the sample does not just include those easiest to reach (such as retirees). Once contacted, interviewers use several techniques – such as asking for the person with the next birthday – to ensure that the sample also does not bias those who most often answer the household phone (statistically, more often women). Those who agree to participate may receive follow-up phone calls or informational mailing to ensure their commitment. In addition, more targeted phone calls may be made to encourage participation among people of demographics less like to the actual polling event (such as those with less education, or those living in remote areas).[15]

While there are no ceilings to the maximum number of participants that could be included in a Deliberative Poll, most polling experiments have had between 130 to approximately 450 participants. Participants are also typically paid between 75-200 $US per-day to partake in the deliberation event. While potentially costly to sponsoring organizations, the use of a monetary incentive encourages people of lower socioeconomic status to attend, allowing for a more heterogeneous participant pool and a better representative sample of the larger public.[16]

How it Works: Process, Interaction, and Decision-Making

While an ordinary poll is designed to show what the public actually thinks about a set of issues, a Deliberative Poll is designed to show what the public would think about those issues if given time for earnest reflection and access to more complete information. Because the focus is on constructing a sample of a hypothetical public attitude, however, the actual Deliberative Polling event has to be highly structured and well organized. Conveners must have access to extensive technical expertise to recruit participants, compose background-briefing materials, and enter and analyze resulting survey data. They also must have sufficient civic resources to pay and train aides and moderators able to ensure a nonpartisan approach to deliberations and discussions, organize the event itself, facilitate media coverage, and disseminate the polling results.[16]

The distinctive procedural features of this polling method include a random sample, balanced background materials, and a series of formal surveys or questionnaires. While the length of deliberation during the actual ‘event’ may vary, in general, deliberations occur over of the course of one or two days. Prior to the event, prospective participants are required to fill out an initial questionnaire and, once returned, are sent background materials that present a range of information and perspectives on the questions under consideration.[17]

During the day(s) of deliberation, participants are informed they are being assembled as a representative cross-section of the larger public. Ideally, providing this information to participants encourages them to adopt an attitude more orientated towards consideration of the potential public, rather than personal, benefits that might result from a particular policy option. That said, appealing to narrower reasons in advocating a specific policy position is not illegitimate. The content of what participants are allowed to say in group-sessions is not restricted, nor is the language in which opinions are articulated. They are also not instructed to stick to the use of logical argument, refrain from personal narratives, or to refer only to the ‘common good’.[18] There are also no formal limits placed on how long an individual may speak, although moderators are instructed to ensure that all participants are allowed to have a voice in deliberations.

Generally speaking, the process of conducting a Deliberative Poll, including the proceedings on the day(s) of deliberation, can be broken down into a five-step process:[19]

1. A random, representative sample of the public is contacted and asked to provide feedback on an initial questionnaire aimed at evaluating the knowledge, perceptions, and preferences of the general public on a specific question.
2. Another random representative sample of citizens are contacted and asked to participate in a ‘deliberative event’ to be held over the course of one or two days.
3. In the lead up to the event, selected participants receive balanced briefing materials vetted by a panel of experts pertaining to the topics to be discussed.
4. On the day of the deliberative event, participants are randomly assigned to small groups with trained moderators. During the group-sessions, the participants are encouraged to develop questions to pose a panel of experts and policymakers at a plenary session held towards the end of the event.
5. The event concludes with all participants filling out a second questionnaire intended to capture their considered opinions on the topic at hand. The results of the first poll are compared to the final poll, and any changes in opinion are measured and analyzed by trained pollsters. In most cases, the findings of the final survey are disseminated to the public through media coverage and, in some cases, form the basis of further debate and deliberations by politicians and policy-makers.

Influence, Outcomes, and Effects

The opinion data produced from Deliberative Polling experiments have been used to various ends – depending on the original agenda of sponsoring organizations, the extent of media coverage, and whether or not the events results are connected directly to political policy-making processes.[20] In most cases, however, the most productive outcome of Deliberative Polling events has been the perceived strength and accuracy of the survey data when compared to other polling methods. Given that citizens are articulating ‘consider judgments’ this data has been considered more legitimate by policy-makers attempting to gauge public support for different, often difficult, policy decisions.

As a survey method, the opinion projection data obtained through Deliberative Polling techniques are intended to provide a more accurate representation of public sentiment on a given issue. The accuracy of the data obtained when comparing the initial ‘baseline’ survey to the data obtained from the final ‘exit’ survey following deliberations has been found to be enhanced when deliberations are prolonged over multiple days rather than just a single one.[21] Nevertheless, in both single and multi-day events the representative microcosm of participants has been found to be highly representative, both attitudinally and demographically, as compared to census data about the entire population.

Almost every Deliberative Polling experiment conducted thus far has also resulted in a number of large and statistically significant changes of opinion following the deliberative event. Considering the results of all Deliberative Polling experiments together, on average about two-thirds of the opinion items have changed following deliberation.[22] Consequently as an opinion poll, the considered judgments arrived at following deliberation have been argued to be superior (and are often different from) the expressed attitudes solicited through conventional polls.

Examples

To date, a number of Deliberative Polling experiments have been conducted with varying degrees of scope for the complexity of the public issues being discussed, the scale of deliberative events (including number of participants, and ties to future policy outcomes. Some notable recent cases include:

• Zeguo, Wenling City, China (April 2005 and February 20 08):

The government of Zeguo used a Deliberative Polling technique to garner public feedback on a series of difficult budgetary allotment decisions that needed to be made for proposed infrastructure projects. Of the entire population of Zeguo, 275 people were randomly selected to participate in surveys and deliberations taking place throughout the months of March and April, 2005. On April 30, 2005 the results of the poll were presented to Zeguo’s local People’s Congress for further debate and deliberation by local politicians – a majority of whom voted to fund the top twelve projects ranked in the Deliberative Poll. After encouraging results from the first Poll held in 2005, a second was held in the Zeguo Township in February 2008 and opened the entire budget to participatory feedback.[23]

• Marousi, Greece (June 2006):

The PASOK, one of Greece’s two major political parties, decided to apply a Deliberative Polling method to select its candidate for mayor in the City of Marousi. A representative sample of 153 citizens from the area were asked to come together and deliberate over six possible candidates, consider their records, pose questions, and at the end provide an advisory survey that ended up constituting the PASOK’s official candidate decision. This was the first time an organization employed Deliberative Polling in order to increase the democratic legitimacy of closed party primaries.[24]

• Omagh, Northern Ireland, United Kingdom (January 2007):

A random sample of parents representing the entire province was brought together to deliberate public educational policy. Held on January 27, 127 parents – both Protestant and Catholic – came together to deliberate in small-group discussions and plenary sessions with panels of experts. This was the first Deliberative Poll ever to be used in a so-called ‘deeply divided society’.[25]

• Sofia, Bulgaria (April 2007):

A national Deliberative Poll was convened by the Bulgarian government to survey considered opinion on changes in policy towards the countries Roma population, issues of crime, and education. The survey sampled 255 residents drawn from around the country and found a statistically significant change towards support for adopting a more integrationist perspective on these issues.[26]

• San Mateo County, California, United States (March 2008):

Sponsored by a coalition of civic groups in the county called ‘Threshold 2008’, a Deliberative Poll was used to survey opinion on the issue of housing shortages. From March 15-16 a weekend sample of 238 participants considered competing policy perspectives, questioned experts, and provided input into a Countywide Assembly on Housing Choices.[27]

• Brussels, Belgium (May 2009):

A deliberative policy organization, 'EuroPolis', brought together 350 citizens from across the 27 European Union member states for a three-day dialogue in which participants considered issues of climate change and immigration in the EU. Deliberating in 21 languages, participants discussed issues, read detailed briefing materials, and questioned competing experts and politicians. Occurring just prior to EU parliamentary elections, the result was the first ‘European-wide’ Deliberative Polling event.[28]

• Porto Alegre, Rio Grande du Sol State, Brazil (June 2009):

A Deliberative Poll was held to solicit state-wide opinion on the issue of career reform in the civil service as the government sought to move away from a ‘years in service’ criterion for promotion. The polling event was held over two-day period, from June 5-7, with participants engaging in small group discussions and dialogue with competing policy experts. The results of the poll were presented to the state government on August 25, and will inform the government’s legislative proposals during the next session.[29]

Analysis and Lessons Learned

The Deliberative Polling technique remains in a relative stage of infancy as a method for public consultation. The exact methods of the poll have often been refined and modified to suit particular questions, events, or contexts with each version having distinct advantages and disadvantages.

In analyzing the results of various Deliberative Polling experiments completed to date, Fishkin has argued the there are four key findings. First, participants’ opinions at the end of the process often substantially change from the beginning in comparison with control groups or other surveys of the population. Second, given the information provided to participants prior to deliberation, those who participate are often more informed after the polling process is completed. Third, these changes of opinion are, in most cases, associated with processes of learning as participants begin to alter their views in response to increased information on a topic. Finally, the emphasis placed on deliberation and discussion during the polling process has had the effect of increasing citizens’ feelings of efficacy and engagement with the political process as a whole.[30]

Fishkin’s findings do differ depending on the level and scope of a given Deliberative Polling experiment. When Deliberative Polling has been used on the national level, it has had the advantage of creating a fairly diverse microcosm of an entire country, ready to confront trade-offs and divisive issues, and providing a guide to others looking to follow informed opinion. This has been particularly true in instances, such as Manchester in 1994, where the polling event is combined with a national broadcast and extensive media coverage.[31] Through widespread dissemination of polling results, citizens who did not participate in deliberations have a chance to access the conclusions and key concerns of deliberators and use them as a guide for their own political behavior. Such events are incredibly expensive, however, and most often Deliberative Polls have been far less ambitious and benefit from operating on a more local level where the face-to-face interactions fostered in the poll can be followed up with broader and more informal discussions at a later date.[32]

However, where Deliberative Polling events are widely covered by the media many critics have argued that the final survey data produced by the poll by be seriously flawed. As Robert Shapiro points out, for instance, The Deliberative Poll can only project what public opinion (more broadly construed) can look like under certain tightly controlled conditions.[33] Everett C. Ladd has added to skepticism about the accuracy of survey data obtained through multiple polling events, by pointing out that the judgements arrived at by participants in Deliberative Polls are likely to be influenced by the ‘Hawthorne effect’.[34] Under controlled conditions subjects have been known to improve in the aspect of their behavior being experimentally measured simply in response to the fact that they are being studied, and not in response to any particular experimental manipulation. In Deliberative Polling events – many of which have received a fair amount of media attention – there has been considerable worry that the judgements reached by participants may display biases of precisely this type, with subjects responding to the spotlight rather than increased knowledge about the subject.[35]

Rather than representing an accurate picture of considered public opinion, Fishkin has also argued that Deliberative Polling might also achieve the creation of a counterfactual opinion – that is, that poll results can represent a projection that has recommending force when disseminated to the broader public. Put this way, a counterfactual representation of public opinion might suggest to the rest of the population some conclusions about a specific issue that they ought to take seriously because it comes from a relatively trustworthy proxy.[36] This is relatively different use of the method than the one first recommended by Fishkin in attempting to correct conventional opinion polls, but it is also a role recent experiments in places such as Greece, Brazil, and China have pushed Deliberative Polling towards.

See Also

Wenling City Deliberative Poll 

The Center for Deliberative Democracy at Stanford University

References

[1] James Fishkin and Cynthia Farrar, "Deliberative Polling: From Experiment to Community Resource," in The Deliberative Democracy Handbook: Strategies for Effective Civic Engagement in the 21st Century eds. John Gatsil and Peter Levine (San Francisco: Jossey-Bass, 2005), 71-72.

[2] Center for Deliberative Democracy - Deliberative Polling (Accessed Nov 9, 2009) 

[3] James S Fishkin. 1988. ‘The Case for a National Caucus: Taking Democracy Seriously.” in Atlantic Monthly (August). p. 18

[4] James Fishkin. 2006. 'Realizing Deliberative Democracy: Strategies for Democratic Consultation' in Ethan Leib and Baogang He (eds.) The Search for Deliberative Democracy in China. New York: Palgrave MacMillan. pp. 44-45.

[5] James Fishkin and Cynthia Farrar 'Deliberative Polling: From Experiment to Community Resource.' p. 73.

[6] James S Fishkin. 1988. ‘The Case for a National Caucus: Taking Democracy Seriously.” in Atlantic Monthly (August): 16-18.

[7] James S. Fishkin. 1991. Democracy and Deliberation: New Directions for Democratic Reform. New York: Yale University Press.

[8] James Fishkin and Cynthia Farrar 'Deliberative Polling: From Experiment to Community Resource.' p. 71.

[9] Robert C. Luskin, James S. Fishkin and Roger Jowell. 2002. ‘Considered Opinions: Deliberative Polling in Britain’ in British Journal of Political Science (32) pp. 457-460.

[10] James S. Fishkin and Robert C. Luskin. 1996. ‘The Deliberative Poll: A Reply to Our Critics.’ in Public Perspective 7(1): pp. 46.

[11] Robert C. Luskin, James S. Fishkin and Roger Jowell. 2002. ‘Considered Opinions: Deliberative Polling in Britain’ in British Journal of Political Science (32) pp. 455-456.

[12] James Fishkin and Cynthia Farrar 'Deliberative Polling: From Experiment to Community Resource.' p. 75.

[13] CDD: Deliberative Polling Toward a Better Informed Democracy - Executive Summary (Accessed Nov 9, 2009)

[14] James S. Fishkin Faculty Profile, Department of Communication, Stanford University (Accessed Nov 9, 2009)

[15] James Fishkin and Cynthia Farrar 'Deliberative Polling: From Experiment to Community Resource.' p. 74.

[16] 16.0 16.1 James Fishkin and Cynthia Farrar 'Deliberative Polling: From Experiment to Community Resource.' p. 74.

[17] James S. Fishkin. 1996. ‘The Televised Deliberative Poll: An Experiment in Democracy.’ in The Annals of the American Academy of Political and Social Science (546). pp. 133-136

[18] James Fishkin and Cynthia Farrar 'Deliberative Polling: From Experiment to Community Resource.' p. 74.

[19] CDD: Deliberative Polling - What is Deliberative Polling? (Accessed Nov 10, 2009)

[20] CDD: Deliberative Polling Toward a Better Informed Democracy - Executive Summary (Accessed Nov 9, 2009)

[21] Robert C. Luskin, James S. Fishkin and Roger Jowell. 2002. ‘Considered Opinions: Deliberative Polling in Britain’ in British Journal of Political Science (32): 455-460.

[22] James Fishkin and Cynthia Farrar 'Deliberative Polling: From Experiment to Community Resource.' p. 76-77.

[23] James Fishkin, Baogang He and Alice Siu. 2006. 'Public Consultation through Deliberation in China' in Ethan Leib and Baogang He (eds.) The Search for Deliberative Democracy in China. New York: Palgrave MacMillan. pp 230-233.

[24] CDD: Deliberative Polling - Greece (Accessed Nov 8, 2009)

[25] CDD: Deliberative Polling - Northern Ireland (Accessed Nov 8, 2009)

[26] CDD: Deliberative Polling - Bulgaria (Accessed Nov 10, 2009)

[27] CDD: Deliberative Polling - California (Accessed Nov 9, 2009)

[28] CDD: Deliberative Polling - European Union (Accessed Nov 9, 2009)

[29] CDD: Deliberative Polling - Brazil (Accessed Nov 9, 2009)

[30] James Fishkin and Cynthia Farrar 'Deliberative Polling: From Experiment to Community Resource.' p. 76.

[31] James S. Fishkin. 1996. ‘The Televised Deliberative Poll: An Experiment in Democracy.’ in The Annals of the American Academy of Political and Social Science (546). pp. 133-137.

[32] Robert C. Luskin, James S. Fishkin and Roger Jowell. 2002. ‘Considered Opinions: Deliberative Polling in Britain’ in British Journal of Political Science (32) pp. 480-487.

[33] Robert Y. Shapiro. 1996. 'Review: The Voice of the People: Public Opinion and Democracy by James S. Fishkin.' in The American Political Science Review 90(3). p. 625.

[34] Everett C. Ladd. 1996. ‘Fishkin’s ‘Deliberative Poll’ Is Flawed Science and Dubious Democracy.’ in Public Perspective 7(1) p. 42.

[35] Everett C. Ladd. 1996. ‘Magic Town: Jimmy Stewart Demonstrates the ‘Hawthorne Effect’.’ in Public Perspective 7(3). p. 16.

[36] James Fishkin. 2006. 'Realizing Deliberative Democracy: Strategies for Democratic Consultation.' p. 45-46.

External Links

Center for Deliberative Democracy - Deliberative Polling

EuroPolis 

Threshold 2008

Notes