Sunday 20 August 2017
   
“I cannot prevent the French from being French.” - Charles de Gaulle
 
Another April surprise? A quick assessment of polls
By Jocelyn Evans, Gilles Ivaldi
23 April 2012 | Polls & Forecasts | 952 words
Now that the official results of the first round are known, it is time to reflect on how accurate voting intention polls have been — both in an absolute sense and as compared with their performances in 2007.

Did pollsters do better or worse than in 2007?

Five years ago, the horse race polls conducted in the final days of the campaign correctly predicted the exact order of the four main candidates (Sarkozy, Royal, Bayrou, Le Pen). Together with the absence of a ‘huge surprise’ –such as Le Pen in 2002 or Balladur in 1995–, this superfecta forecast led many to consider that the 2007 polls had been reasonably good. Yet, this view must be nuanced. First, the polls largely missed the margin between Sarkozy and Royal, mainly because the former’s score was underestimated by 3 per cent on average. Second, pollsters proved yet again unable to forecast accurately the FN vote: learning from the lessons of 2002, most of them adjusted Le Pen’s numbers upwards, which led to an overestimate of over 3 per cent.

Did pollsters pop open bottles of champagne last night? Like in 2007, their last week forecasts have predicted the final order of the five main contenders across the finish line. Taking the average of the seven polls published in the campaign’s final week gives a very accurate picture of the balance of forces between the two frontrunners with a 0.8-point variation in both Hollande and Sarkozy. This year the main source of error is found in the two peripheral ‘protest’ candidates: Mélenchon (-2.7) and Le Pen (+2.1).


Final week polls and official results

Pollster Date Hollande Sarkozy  Le Pen  Bayrou  Mélenchon 
OpinionWay 17/04/2012 27,5 27,5 16 10 13
LH2 18/04/2012 27 26,5 15,5 10 15
IPSOS 19/04/2012 29 25,5 16 10 14
CSA 19/04/2012 28 25 16 10,5 14,5
BVA 19/04/2012 30 26,5 14 10 14
TNS Sofres 19/04/2012 27 27 17 10 13
IFOP 20/04/2012 27 27 16 10,5 13,5
Mean 27,9 26,4 15,8 10,1 13,9
CAWI 27,3 27,3 16,0 10,3 13,3
CATI 28,2 26,1 15,7 10,1 14,1
1st round* 28.6 27.2 17.9 9.1 11.1
  Difference 0.7 0.8 2.1 -1.0 -2.7

*Source: Ministry of Interior



Overall, a ‘crude’ measurement of polling error can be taken from the sum of absolute errors for all candidates: in 2007, the total of errors was 12.2 across all twelve contenders. This year the same calculation gives an eight-point error for the ten candidates in the presidential race. In sum, the global performance by France’s pollsters has improved compared with 2007. Pollsters could –and will– legitimately claim that even the largest differences are within the margin of error.


The Marine Le Pen vote

Key to this year’s election was of course the ability by pollsters to give an accurate estimate for Marine Le Pen: after nearly 40 years of reign, her father finally stepped down from party leadership in early 2011. There have been speculations about the electoral impact of the so-called ‘de-demonization’ strategy pursued by his daughter. During the campaign, pollsters made no secret of the fact that their raw data had less of the ‘social desirability bias’ traditionally associated with the FN vote in France.

This is corroborated by the election returns: whilst Marine Le Pen has fallen short of replicating the 2002 political ‘coup’ of progressing to the run-off, she has achieved the best score ever for the extreme right in France, surpassing even the total number of votes won by her father in the second round of the 2002 election (6.4 million votes in 2012 compared with 5.5 million ten years ago).
With an average score of 15.8 per cent in the final week, polls have failed to measure the exact level of support for the FN leader, as have in fact most commentators. The symbolic significance of this is hardly debatable and was probably further amplified by some pollsters unwisely giving Le Pen up to 20 per cent of the vote in their early evening estimates. From a methodological perspective, however, the final week polls have managed to reduce the level of uncertainty in predicting the extreme right: in absolute terms, their average 2012 error (2.1) is smaller than both their 2002 and 2007 misforecasts (respectively 3.3 and 3.4).


CATI vs CAWI

In a previous post, we provided some empirical evidence for a consistent bias in internet polls (CAWI) relative to telephone polls (CATI). An updated analysis of all 119 horse race polls published since June 2011, controlling for the time-period in which they took place, confirmed that Hollande tended to score around 2 per cent less in internet polls, and Marine Le Pen 1.7 points higher

One first observation is that internet polling seems to help break the ‘spiral of silence’ that prevents voters to disclose their preferences for protest or extreme political forces. In yet another plea for transparency, pollsters should agree to publish their raw data to allow for a more scientific assessment of such effects which are anything but trivial. In the light of the massive overestimation of Marine Le Pen’s vote by some of the first exit polls of the evening, most polling organisations are hardly in a position to argue that their reticence about releasing their adjustments serves a public good.

Differences have been waning in the most recent surveys –probably due to pollsters looking at each other’s forecasts to smooth their own numbers. Such differences have become negligible in the final week, with the exception of the numbers given for the two finalists: in anticipating a Hollande / Sarkozy tie, internet polls have failed to show the real margin between the frontrunners, mostly because of their underestimating the PS candidate (1.3).


Predicting the left / right balance of forces

Finally, contrary to 2002, the left versus right balance was almost perfectly estimated in 2007 (less than half a percentage point for the left bloc as a whole). This year, polls were anticipating a surge in electoral support for the left: in the final week, the forecast for the total score of the five candidates representing the left camp was up to an average 46 per cent.

The final results show that such wave of popular enthusiasm was overestimated (-2.2), mostly because of the amplification of the Mélenchon phenomenon in the polls.

Back


Welcome to '500Signatures', for analysis and commentary on French politics and elections

This blog is produced by Jocelyn Evans (University of Leeds) and Gilles Ivaldi (University of Nice)

 
PREVIOUS POSTS
Politicizing terror: terrorism and the 2017 presidential race
Hollande’s calculation behind the French socialist presidential primary
Is the French 2017 presidential battle already over?
Estimating Marine Le Pen’s 2017 presidential vote share
Walking a fine line? Hollande and the French Left
All roads lead to Rome: French parties on the way to the 2017 presidentials
Ils ne passeront pas – the stemming of the FN tide in the regional run-offs
The FN on the threshold of regional government
Regional elections and the anti-Muslim backlash
Politics in a time of war?
A right-wing landslide but no far-right tsunami: the departmental election run-off
The Front national is not France’s first party
What to expect in next month’s French departmental elections
Departmentals 2015: the new French elections no-one seems to care about
Beta-testing social-liberalism 2.0
France’s new earthquake election? The FN in the European elections


57
posts have been published
since 10 January 2012

Show all posts
 
ABOUT THE AUTHORS

Jocelyn Evans [@JocelynAJEvans] is Professor of Politics at the University of Leeds

Gilles Ivaldi is a CNRS researcher in political science based at the University of Nice

 
BOOK
 
RECOMMENDED






 
CATEGORY
 
DATA

- Forecasting the FN vote in Second-Order elections (updated 12 May 2014)

- Forecasting the FN vote in Second-Order elections (Jan. 2014)

- Polling scores by polling type (CATI v CAWI) (updated 20 April 2012)

- Estimating Marine Le Pen's vote in the 2012 presidentials: an experiment (November 2011)

- Data for the 2011 expert forecast survey (in CSV file)

 

 


 
Last modified on Monday 25 April 2016
Copyright Gilles Ivaldi - @2012-2014