It’s not quite the question on refugees as such that I was looking for, but if I had not been behind in Pollytics blog reading I would have seen this post reporting an Essential Research poll last year on the size of the refugee program. The table below, taken from Pollytics, shows that a small majority in late July and early August 2008 were opposed to the program at its current size.
The same poll found that 62% of respondents thought that previous tough policies on asylum seekers were about right or not tough enough.
Overall, it does look like voters have issues with the refugee program as such, though some of those who are happy with the size of the program seem also to favour tough action against self-selecting refugees.