For as long as we’ve had media and political parties, the former has been accused of partisan bias by the latter. In both Canada and the US, those on the right often complain that the media has a “liberal bias” while those on the left complain of “corporate media” or “conservative bias.” Is there actually bias or is this just a story partisans tell themselves?

Social media sites and the big data they produce offer new, experimental ways to measure nebulous concepts like bias. I’ve tried to uncover signs of partisan bias in Canada’s Parliamentary Press Gallery using Twitter followers.

My analysis used an approach developed by Stephen Tapp that compares numbers of Twitter followers to infer the ideology of think tanks (Policy Options, January-February 2015). Tapp used the same approach to infer the ideology of economists (reported by Jason Kirby of Maclean’s magazine).

If A’s followers are mostly X, with few Ys among them, there is a good chance A’s thinking, at least as expressed in tweets, is decidedly X. So an economist with more followers that match a “right-wing” think tank than followers that match a “left-wing” think tank might be inferred to be more right wing than left. Of course, there are caveats to which I will return below.

Tapp’s analysis suggests this methodology could be used to test for other biases — such as those of political journalists.

I turned to the official listing of the Parliamentary Press Gallery. I focused on journalists who I believe have reach outside the confines of Parliament Hill, so I ignored the Hill Times, iPolitics and CPAC. I also excluded foreign and Aboriginal correspondents and freelancers. I admit these are subjective decisions. Finally, I included only reporters and columnists with more than 5,000 Twitter followers, to keep the list manageable. I didn’t search every person on the list as it includes non-journalists like camera operators, so it is possible I overlooked someone who should be included. I am quite certain they will let me know.

Tapp’s approach requires two “anchor” Twitter accounts on opposing ends of the spectrum. I initially hoped to use the Twitter accounts of Stephen Harper and Justin Trudeau as my anchors, but immediately ran into trouble because they have too many followers for Followerwonk to handle.

So I needed to find benchmarks with smaller but similar numbers of Twitter followers. Politwitter.ca publishes a useful ranking of Canadian MPs by twitter followers that was helpful in my analysis. Other than the selection of MPs with similar numbers of followers, the choices are also subjective. I have no doubt others would, and will, make different choices. The required data crunching is not difficult.

Tapp’s article describes the details, but essentially how it works is that you look for Twitter followers of the Conservatives who do not also follow the Liberals (they are deemed “Conservative” followers), and followers of the Liberals who do not also follow the Conservatives (“Liberal” followers). Then you look at each reporter’s Twitter account and compare his or her relative percentages of Conservative versus Liberal followers. The difference between the two is a measure of the reporters’ inferred partisanship.

Figures 1 and 2 illustrate the approach using Michelle Rempel and Ralph Goodale as our Conservative and Liberal anchors and comparing their followers with those of journalists Joan Bryden and Jason Fekete.

In the comparison with Joan Bryden (figure 1), Rempel has 9,418 unique followers and Goodale has 9,534 unique followers. The two of them share 1,137 followers. Joan Bryden shares 699 followers with Michelle Rempel and 1,274 followers with Ralph Goodale. When we adjust for the fact that Goodale has more overall followers than Rempel, we see that Bryden has an 11.5 percent overlap with Rempel and a 19.8 percent overlap with Goodale. The difference is 8.3, which is Bryden’s “Liberal” partisanship score. Using the same calculations (figure 2) produces a 12.0 percent- “Conservative” partisanship score for Jason Fekete (a size-adjusted 21.3 percent of his followers’ overlap with Rempel minus the 9.3 percent overlap with Goodale).

Figure 3 shows the full press gallery analysis using Michelle Rempel as the Conservative anchor and Ralph Goodale as the Liberal anchor. Both Rempel and Goodale are active partisans for their respective parties. Goodale has a higher rank within the Liberal caucus than Rempel does within the Conservative caucus, but they have similar numbers of followers and both have built their Twitter networks largely through politics.

The Rempel versus Goodale graph skews heavily Liberal and both tails of the distribution have a number of scores above 5. Jason Fekete’s unusually high Conservative score is undoubtedly due to his having recently worked in Calgary (at the Calgary Herald) — the city from which Rempel hails.

To ensure that these results are not due to things that are unique to Rempel and Goodale, I repeated this exercise two more times with different (subjective) choices for Conservative and Liberal anchors.

In my second analysis, I used James Moore as my benchmark Conservative and Marc Garneau as my benchmark Liberal (figure 4). Moore and Garneau might be expected to have a fairly large crossover of followers (both partisan and otherwise) beacause of the former being a prominent economic minister (Industry) and the latter having a loyal following due to his former prominence as a Canadian astronaut.

Unlike the Rempel versus Goodale comparison, the Moore versus Garneau graph evenly splits reporters into left and right: 20 fall to the left and 19 to the right. And, other than outlier Hannah Thibedeau on the left, few reporters have scores above 5 (a score of 5 would indicate that 5 percent more of the reporter’s followers matched those of Moore or Garneau — but not the other).

In my third comparison I used Lisa Raitt as my Conservative anchor and Scott Brison as my Liberal anchor (figure 5). Again, the graph skews heavily Liberal, though the average scores, particularly on the Conservative side, are below 5.

If you compare the rank order of various reporters, you find that Chris Hall, Tim Harper, Hannah Thibedeau, Terry Milewski, Glen McGregor and Aaron Wherry rank in the Liberal top third across all three comparisons, while Paul Vieira, David Akin, Vassy Kapelos and Jason Fekete rank in the Conservative top third across all three comparisons. Laura Payton, Robert Fife, Tonda MacCharles and John Ibbitson are all in the middle third for all three comparisons.

Finally, I wanted to combine these three comparisons into a single measure. The correlations between the rankings of two of the three pairings were reasonably high: 0.790 between Moore/Garneau and Raitt/Brison; 0.716 between Raitt/Brison and Rempel/Goodale; but only 0.353 between Moore/Garneau and Rempel/Goodale.

I tried a number of different ways of combining the three scores and found the results were very closely correlated, and so I settled on adding up the numbers of overlapping followers for each of the comparisons and rescoring each member. For each journalist I added up the unique shared followers with Moore, Rempel and Raitt to get their Conservative score. Clearly this will result in some double counting”ˆ— for example, if someone followed Moore, Raitt and Rempel but not Garneau, Goodale or Brison, this would add 3 on the Conservative side and 0 on the Liberal side when a pure score would be only 1. Still, it would be the same problem on both sides of the spectrum, so while the raw score might not be pure, the order of magnitude should be indicative.

I also compared average scores across the three comparisons. The graph was virtually identical to the result of using the previous methodology, with only one change in rank order — Terry Milewski and Michael Den Tandt switch rank order. And the correlation between the two was 0.99.

The resulting graph (figure 6) shows a strong Liberal skew with the following press gallery members scoring above 5 on the Liberal side:

• Hannah Thibedeau, CBC (17.6)
• Tim Harper, Toronto Star (16.0)
• Chris Hall, CBC (13.7)
• Joan Bryden, Canadian Press (13.6)
• Terry Milewski, CBC (11.4)
• Michael Den Tandt, National Post (11.1)
• Allison Crawford, CBC (11.3)
• Julie Van Dusen, CBC (10.6)
• Aaron Wherry, Maclean’s (9.9)
• Stephen Maher, National Post (9.5)
• Glen McGregor, Ottawa Citizen (9.0)
• Althia Raj, Huffington Post (7.6)
• Rosemary Barton, CBC (5.9)
• Kathleen Harris, CBC (5.8)
• Heather Scoffield, Canadian Press (5.7)
• Laura Payton, CBC (5.4)

And the following scoring about 5 on the Conservative side:

• Jason Fekete, Ottawa Citizen (16.9)
• Vassy Kapelos, Global TV (11.1)
• David Akin, Sun TV (9.1)
• Paul Vieira, Wall Street Journal (6.1)

Among Liberals we have reporters and columnists from a number of media properties, with a preponderance from the CBC. Among Conservatives, two made their living reporting on politics in the Conservative heartland of Alberta before moving to Ottawa, one works for the Conservative-leaning Wall Street Journal and the other was recently a journalist at the now defunct, Conservative-leaning TV station Sun News Network.

Before you draw your own conclusions, it is worth asking how well this methodology measures partisanship. I consider myself a devoted Harper Conservative and my scores on the various indexes (I admittedly have only 1,646 Twitter followers) came out as follows:

• Moore/Garneau: 24.5 Conservative
• Rempel/Goodale: 31.9 Conservative
• Raitt/Brison: 12.1 Conservative
• Total score: 67.5 Conservative

So before any reporter thinks they have been outed as a hard-core partisan, I would note that no score comes close to that of someone who actually is.

And while this is an interesting tool based on a valid insight, it’s not hard to imagine how factors other than partisan affinity might result in a heavy skew in a reporter’s followership. Remember when Terry Milewski was driving the Chrétien Liberals nuts? I’m pretty sure that if Twitter had existed then, his follower numbers would have leaned Conservative, or at least much more Conservative than they do now. Why? Not ideological or partisan affinity, but simple confirmation bias (such as “I always knew Chrétien is an XXX and Milewski’s proving it!”).

This is not entirely speculative. Remember that the theoretical underpinnings of the “birds of a feather flock together” argument are psychological: the fact that we enjoy having our ideological or partisan identities supported and dislike having them challenged, and therefore tend to seek out those sources that deliver, is merely one variety of the broader phenomenon of confirmation bias.

And there is also location bias: Jason Fekete and Vassy Kapelos were based in Alberta prior to moving to the Parliamentary  Press Gallery — and this undoubtedly resulted in their having more followers similar to Conservatives in general and Michelle Rempel in particular. And it would make sense that Marc Garneau would have larger overlaps with Quebec-based, or French-language, reporters.

I have no doubt that others will do their best to come up with other biases. Still, I think these rankings, while clearly imperfect, are quite interesting.

Photo: Shutterstock