Monday 11 December 2017
   
“I cannot prevent the French from being French.” - Charles de Gaulle
 
Facts and artefacts in French election polling: a plea for transparency
By Jocelyn Evans, Gilles Ivaldi
10 February 2012 | Polls & Forecasts | 1263 words
Since the infamous 1936 Literary Digest poll misforecasting a famous victory for US Republican Alf Landon over the Democrat incumbent Franklin Roosevelt, people have had good reason to be skeptical about pre-election polls. Indeed, even by 1948, a more methodologically refined approach to survey research championed by Angus Campbell and Robert Kahn had proved its worth in correctly backing Harry Truman in that year’s presidential race, unlike pollsters, who had all their money on Thomas Dewey. In France, the more conceptual critique of mass opinion polling as a “science without a scientist” by sociologist Pierre Bourdieu has nurtured immense distrust in what American political scientist Benjamin Ginsberg alternatively referred to as a “subtle instrument of power”.

Even below the heady heights of critical sociology, pre-election polls have constantly been the subject of sharp criticism in the French political arena for simple empirical failure. In 1995, pollsters came under fire for pre-emptively crowning centre-right candidate and outgoing Prime Minister Edouard Balladur as the new Head of State. Seven years later, scathing criticism was directed at polls once again for not having anticipated Lionel Jospin’s elimination from the 2002 presidential race. It is only because things went “according to plan”, i.e. a straight Left-Right run-off, in 2007 that the enduring lack of accuracy in the polls – most notably, the over-estimation of Jean-Marie Le Pen’s eventual score – went almost unnoticed.

The current presidential campaign is not immune to this type of debate. We have already addressed some of these issues in two previous posts on ‘house effects’ and the discrepancies that can be observed across pollsters. Two recent developments in the campaign have turned our attention back to the significance of pre-election polls and the need for some transparency in the methods employed by polling institutes. First, the Council of State (Conseil d’Etat) has confirmed the right of French pollsters to keep their methods for poll adjustment and data weighting under wraps. In their decision, the administrative jurisdiction deemed such practices equivalent to business ‘trade secrets’. This has infuriated left-wing harbinger Jean-Luc Mélenchon who initially brought the case to the Council following a controversial Harris Interactive poll allocating him a measly 3 per cent of the vote, and after urging the institute to disclose their calculation methods. Mélenchon has indicated he will lodge his case at the European Court of Human Rights

The absence of transparency has not escaped the politicians. This very question was central to the provision of a cross-partisan bill proposed in February 2011 to amend the 1977 law on polling, but to date is still pending a vote by the National Assembly. Whilst understandable for commercial reasons, the reluctance of French pollsters to disclose their in-house processes of post-survey weighting contrasts sharply with best practice elsewhere. The British Polling Council – the counterpart to France’s Commission des Sondagesrequires that pollsters make available all weighted and unweighted figures. A half-way house is Italy, which legally requires polling institutes to reveal their sampling methods, non-response rates and post-collection adjustments – but in a survey by Dr Graziella Castro of the University of Salford, only around 7 per cent of institutes actually did so.

Another controversy has been triggered by the publication of a series of polls testing vote intentions were Marine Le Pen not on the ballot paper. Each of these polls provided a different picture of what the first round of the presidentials would look like should the FN candidate fall short of the precious 500 signatures (see chart below). IFOP, for instance, anticipates a tie in the first round between Hollande and Sarkozy, should Le Pen not be present. BVA and IPSOS see a rise in Sarkozy’s vote, but not sufficient to reach his Socialist rival. As Le Monde’s polling blog notes, one reason for IFOP’s more favourable score for Sarkozy is its omission of the smaller candidates, many of them on the Right. But that still does not explain a number of other disparities.


Summary of Polls “without Marine Le Pen”*

*Candidates’ gains under the hypothesis that Marine Le Pen would not be able to run in the first-round


What emerges from this latest discussion is the disagreement among pollsters over the use of internet panel surveys (the so-called CAWI method) in electoral polls. Interestingly, a similar debate had already taken place in March 2011 after Harris Interactive released a much commented internet-based poll showing Marine Le Pen leading the first round of the presidentials with 23 per cent of the vote.

The argument made by detractors of internet samples is that of the specific mode of recruiting panel members via commercial sites, often in exchange for economic incentives as blatant as shopping vouchers and the like. Whilst they might at a pinch be representative of the social structure of the French population – although again, without transparent reporting of sampling frames, quotas and non-response, even this has to be taken on trust – the attitudinal profile of those respondents might differ.

It might also be the case that highly motivated far right sympathizers will seize this opportunity to express their views – a phenomenon not limited to the Internet. Witness, for instance, the attention-grabbing headline from Le Pen’s campaign site that Marine leads the polls amongst iPhone owners. Lastly, it is often argued that internet polls might help overcome the traditional ‘shameful vote’ problem, voters being more prone to reveal their true preferences when seated in front of their computer screen, rather than engaged in conversation with a human being in a CATI setting. One cannot help but draw parallels with individuals’ apparent willingness to engage in the sort of vicious outbursts and slander common to all web discussions that they would shy well away from in less anonymous interactions.

There is empirical evidence that the internet polls consistently bias scores relative to telephone polls. Taking all election surveys since June 2011 asking for vote intentions, and controlling for the time-period in which they took place, our calculations based on 56 polls published since June 2011 indicate that Hollande tends to score around 2.5 per cent less in internet polls, and Marine Le Pen two-and-a-half points higher. Similarly, the gap between Le Pen and Sarkozy – that crucial score that defines whether the run-off becomes another referendum on democracy, as in 2002 – is around 2.5 points narrower amongst internet polls. Going on internet polls, then, the race is tighter, with Marine Le Pen threatening to overtake the incumbent president. Telephone polls suggest at this stage a more likely two-candidate run-off with Le Pen trailing.

Of course, these differences are relative. It is possible that the internet polls represent the ‘true’ (pace Bourdieu) system of social forces at play in the French electorate, and the telephone polls are overselling Hollande at the expense of Le Pen. Furthermore, such a net trade-off is simplistic – the shifts by candidate aggregating to this position are likely more complex, and again due to differences in sampling rates and latent bias.

But this again highlights the current problem. If we cannot know exactly who is answering the question, “If the election were held tomorrow, how would you vote?”, and with what weighting, it is impossible to begin even to assess which scores are factual, which are artefactual, and which are outright fiction. France may not yet be characterised as a sondocrazia like its Southern neighbour, where polls are seen as explicit tools for electoral manipulation and a “guide” to voters; but to the extent that polls do influence voters in their suggestion of the likely winners and losers, the political information which pollsters provide as a public service should at least come with their own instruction manual.

Back


Welcome to '500Signatures', for analysis and commentary on French politics and elections

This blog is produced by Jocelyn Evans (University of Leeds) and Gilles Ivaldi (University of Nice)

 
PREVIOUS POSTS
Politicizing terror: terrorism and the 2017 presidential race
Hollande’s calculation behind the French socialist presidential primary
Is the French 2017 presidential battle already over?
Estimating Marine Le Pen’s 2017 presidential vote share
Walking a fine line? Hollande and the French Left
All roads lead to Rome: French parties on the way to the 2017 presidentials
Ils ne passeront pas – the stemming of the FN tide in the regional run-offs
The FN on the threshold of regional government
Regional elections and the anti-Muslim backlash
Politics in a time of war?
A right-wing landslide but no far-right tsunami: the departmental election run-off
The Front national is not France’s first party
What to expect in next month’s French departmental elections
Departmentals 2015: the new French elections no-one seems to care about
Beta-testing social-liberalism 2.0
France’s new earthquake election? The FN in the European elections


57
posts have been published
since 10 January 2012

Show all posts
 
ABOUT THE AUTHORS

Jocelyn Evans [@JocelynAJEvans] is Professor of Politics at the University of Leeds

Gilles Ivaldi is a CNRS researcher in political science based at the University of Nice

 
BOOK
 
RECOMMENDED






 
CATEGORY
 
DATA

- Forecasting the FN vote in Second-Order elections (updated 12 May 2014)

- Forecasting the FN vote in Second-Order elections (Jan. 2014)

- Polling scores by polling type (CATI v CAWI) (updated 20 April 2012)

- Estimating Marine Le Pen's vote in the 2012 presidentials: an experiment (November 2011)

- Data for the 2011 expert forecast survey (in CSV file)

 

 


 
Last modified on Monday 25 April 2016
Copyright Gilles Ivaldi - @2012-2014