My Australian political identity survey closed this morning, and I will write several posts on the results over the next few days. But first some discussion of how respondents were solicited and the biases that might cause – some of which is in response to questions and criticisms since the survey began.
The first point is that this was not a normal public opinon survey – it was a survey of people willing to identify with with a philosophical political label. The difficulty this poses is in finding such people, who are likely to be a small percentage of the electorate. If a pollster was commissioned to do it I expect it would take tens of thousands of phone calls to produce the 1,200 responses I received in my online survey.
My method was to largely recruit via blogs. Apart from my own, there were links from Andrew Leigh, Andrew Carr, Australian Conservative, Andrew Bartlett, Institutional Economics, the f-rant, Sacha Blumen, Club Troppo, Thoughts on Freedom, Pollytics, Quadrant blog, and Catallaxy.
Blogs were supplemented with my personal networks, via email and my Facebook page.
The advantage of these methods over random sampling is that they are more likely to capture the people most engaged in advocating the ideologies under discussion. This method also takes us a little closer to the other major methodology for understanding political ideologies, which is examining the published views of their advocates.
With a bigger budget (I spent about $40 for the online survey), the results could be improved by advertising the survey to the members of relevant organisations and readers of relevant publications, and perhaps providing a non-online version. The number of conservative respondents was lower than I expected, so maybe internet survey recruitment methods are not good at finding conservatives.
The main other concern is whether these blogs cause an over-representation of a particular strand of thinking within a broader ideology. I’m pretty confident that this won’t be the case for the classical liberal-libertarian blogs. I’m not so sure about social democrats. When the only social democratic-leaning blogs linking to the survey were Andrew Leigh and Club Troppo I was a little concerned that their readers would be more economically literate than I would expect social democrats more generally to be. But there was a surge of social democrat responses when Crikey-sponsored blogs linked to me, so this probably fixes the problem. I wasn’t represented on a green blog, but a quick look at the results suggests that stereotypes will be confirmed, so there is no initial reason to believe that despite their relatively small number (92) they will be unrepresentative.
The biggest obvious problem with the results, however, is the gender balance – or lack thereof. Only 14% of respondents overall are female. Even the most ‘progressive’ of the respondent groups, the greens, are more than 80% male. The 2007 Australian Election Survey does show that women are less interested in politics than men, though not by anywhere near this margin, and have similar (low) rates of internet usage for political purposes as men. As I go through the various ideologies, I will see if there are obvious gender differences in the responses that mean the lack of women could be affecting the results.