Political pollsters who had predicted the outcome of the 2018 Quebec election could have been somewhat nervous as this year’s federal election approached. Their forecasts in October 2018 had been embarrassingly wrong: the polls substantially underestimated the support for the Coalition Avenir Quebec and produced a similarly large overestimate of the support for the Parti Libéral du Québec. Further analyses showed that the polling miss was mostly due to voters changing their minds, but what if it happened again? Even if there was no problem with methods, if a similar polling miss were to happen, wouldn’t it be attributed to pollsters’ methods anyway?
The results of the federal vote on October 21 must have been a relief. The polls were mostly right. There is still room for improvement, however, particularly in the forecasting of changes in voting intention over the course of the campaign. The poll results that get the most attention are the responses to the usual question: Who would you vote for if the election were held tomorrow? But other polling questions, for example about respondents’ levels of certainty and their second choices, could help to predict changes in support for the various parties.
What did the polls tell us?
In order to estimate change in support for the parties over the electoral campaign, I use a statistical procedure that gives more weight to polls that are similar to each other and less weight to “outlier polls,” those whose estimates are different from other pollsters’. The procedure — local regression — uses 40 percent of the estimates around each time point, which gives an estimate of change that is rather sensitive to events. Figure 1 shows the trends in support for each party, together with election results in Canada as a whole, starting from shortly before the election call in September. The dots show the polls’ estimates, positioned at the middle of their fieldwork. The trend lines show the estimated support for each party during the campaign, based on local regression.
In the last polls before election day, the Liberal Party of Canada (LPC) and the Conservative Party of Canada (CPC) had similar levels of support, at around 32 percent. Their numbers had been very stable during most of the campaign, although both were decreasing toward the end. Meanwhile, support for the New Democratic Party (NDP) was growing through the campaign but reached a plateau of close to 20 percent during the last week. Support for the Bloc Québécois and for the Greens was estimated at the same level, at around 7.5 percent each, in the final polls. However, support for the Bloc had been increasing from the beginning of the campaign while support for the Greens was steadily decreasing.
Election results showed that polls had slightly underestimated the support for both the main parties. The LPC got 33 percent of the vote and the CPC, 34 percent. By contrast, the NDP was clearly overestimated in polling, receiving 16 percent of the vote. The Greens got slightly less in vote share than their poll numbers would have suggested, at 6.5 percent, and the Bloc’s vote matched its polling almost perfectly. Overall, the pollsters’ performance was very good and it did not mislead voters. The various poll estimates were generally within the margin of error of the final results, except for the NDP’s.
There is no indication that the “brownface/blackface” event (starting with images released on September 18) and the subsequent controversy around it had any impact on voting intentions. Whether the official English leaders’ debate had an impact may be subject to interpretation. From figure 1, it appears that voting intentions started to change before the debate. However, the debate was followed by an acceleration of the NDP surge and a decline in support for the LPC and the CPC. During the last week, the support for most parties reached a plateau, except for the LPC’s, which seemed to pick up.
Figure 2 shows the picture in Ontario. In earlier elections, there had been a frequent underestimation of the Conservative support in Ontario, and NDP support was often overestimated. This time, the LPC’s support in the polls was 39 percent, and the party got 41.5 percent of the vote; the NDP’s support was estimated at close to 20 percent, but it got less than 17 percent. In short, the polls estimated the Conservative vote accurately while they underestimated the LPC vote; the NDP vote was again overestimated.
Figure 2 clearly indicates that no change in support for any party occurred after the “brownface/blackface” event. However, the official English leaders’ debate seems to have stopped the slide in support for the LPC, which increased its share in the last days of the campaign. The increase in support for the NDP before and after the debate eventually halted, and NDP support reached a plateau in the last week of the campaign.
In British Columbia (figure 3), the situation was quite different. Support for the Conservatives was clearly underestimated there, and the surge in support for the NDP in the final days was slightly overestimated by the polls, at the expense of support for the LPC. Support for the Greens was quite well predicted. The English debate was followed by a 5-point increase in support for the NDP, concurrent with a 4-point decrease in support for the LPC. During the last week, support for both the LPC and the NDP grew slightly at the expense of the Greens. Support for the CPC stayed stable.
Figure 4 shows what happened in Quebec. Although movements in voting intentions had started before the TVA debate, on October 2, this debate seems to have accelerated the drop in support for the LPC and the CPC and the surge of the Bloc. After the second French debate, the Bloc, the NDP and the Greens reached a plateau while support for the Liberals started to rebound. As forecasted, the LPC did slightly better than the Bloc. The Conservatives did slightly better than expected while support for the Greens, but above all for the NDP, was overrestimated.
Our newsletter about the public service.
Nominated for a Digital Publishing Award.
Looking at the three major provinces gives a hint of what happened in the final days or even hours of the electoral campaign. After some movement in the weeks after the debates, support for some of the parties crystallized during the last week. However, this does not fully explain the election results. There likely was some “last-minute” movement among voters. My research, with colleagues, has shown that those who intend to vote for smaller parties may decide in the end either to vote for a major party or not vote at all. This translates into polling estimates that may be too optimistic for the small parties and too pessimistic for the main parties. Also, the so-called main parties may not be the same ones in all regions of the country. The leading parties were the LPC in Ontario and the CPC in BC; support for both was underestimated by the polls. In Quebec, there were two main parties this time, the LPC and the Bloc Québécois; their support was very accurately estimated. On the other hand, NDP support was clearly overestimated in the three provinces.
What can we learn from undecided voters?
We are used to labelling as “undecided” those poll respondents who do not reveal their preferences. However, during the campaign, some pollsters asked respondents who had revealed their preferences whether they were sure of their vote or whether they might change their minds. Their responses provide an additional insight into why the polls may miss the target, by revealing shifts in party support that may occur during the campaign.
Ipsos asked respondents whether they were very certain, pretty certain, not very certain or not certain at all of their choice. Figure 5 shows that the proportion of respondents who were not certain of their choice (“not very certain” combined with “not certain at all”) decreased over the campaign. However, it was always higher among NDP and Green voters.
Léger asked a different question: whether the respondent’s choice was final or could change. Figure 6 shows that the proportion of those who said their choice was not final (a combination of “may change” and “don’t know”) declined. But right until the end, proportions of NDP, Green and, to a lesser extent, Bloc voters who declared that they might change their minds were higher than among voters for the main parties. The results are similar to the Ipsos results.
An additional question in Léger’s October 7-8 poll asked for respondents’ second choice. Answers to this question confirmed the levels of commitment shown in figure 6: NDP, Liberal and Green voters were second choices of each other while the Conservative voters, more than voters of any other party, tended to have no second choice. And supporters of other parties did not tend to have Conservatives as their second choice. Therefore, the Conservative vote appeared strong but could not increase by much.
During future election campaigns, detailed polling results about which respondents are more likely to change their minds may help forecast possible changes in voting trends. When voters who have expressed a preference for a party say they might switch, at least some of them do, and usually from smaller parties to main parties. These shifts can explain the usual overestimation of support for the small parties.
The polls conducted during the 2019 federal election campaign were generally accurate, as they usually are, in fact. The results confirm that there are no major problems with the methods used to estimate the vote. The polls informed the voters pretty well of what was likely to happen. They clearly showed that in Canada as a whole, support for the two main parties was similar. They also showed that the battles were not the same in the different regions of the country. In addition, there was enough variation in pollsters’ estimates to conclude that no herding — when pollsters align with one another — took place.
Researchers have known for a long time that polls tend to overestimate support for the NDP, the Greens and sometimes the Bloc. This happened also in 2019, except with the Bloc, whose support was very well estimated for a third election in a row. My analyses also show that looking at the proportion of voters from each party who say they might change their minds or are not certain of their vote can point to possible overestimation of support for some parties.
Finally, as in almost all the elections held in Canada since 2004, electoral campaigns do matter. Support for the different parties fluctuates during the campaign, and movement tends to occur mostly during the last two weeks.
Do you have something to say about the article you just read? Be part of the Policy Options discussion, and send in your own submission. Here is a link on how to do it. | Souhaitez-vous réagir à cet article ? Joignez-vous aux débats d’Options politiques et soumettez-nous votre texte en suivant ces directives.