Archive for the 'Dubious research' Category

Is the ANU better than Stanford?

On the front page of The Australian‘s print edition today a headline reads:

ANU pips Stanford

The internet headline was a little less counter-intuitive, but the story the same. It’s a reference to the 2008 Times Higher Education rankings which puts the ANU at 16th in the world, and Stanford at 17th.

Now the ANU is a perfectly respectable university. But the THE rankings have been widely criticised, and results like this will not help the case for the defence.

The biggest criticism of the THE is their heavy reliance on subjective measures. 40% of the ranking is based on academic peer review, which is done by emailing tens of thousands of academics with an internet survey. The response rate is typically poor, and the response quality doubtful. One potential benefit of rankings is that use of objective information can correct impressionistic views of universities, but this method tends to reinforce the latter. The Shanghai Jiao Tong rankings use purely objective measures (though the weightings are subjective; I don’t think there is any way to make these objective). But while the THE rankings are dubious on this measure, this isn’t what’s dividing the ANU and Stanford. They both get the maximum score of 100.
Read the rest of this entry »

Social capital confusion

Commenter Jarryd saw Postmodern Conservatism in Australia authors Geoff Boucher and Matthew Sharpe give a presentation based on their book, and came away unimpressed:

From memory the section we read was exploring the damaging affect of post modern conservatism and the actions of “neoliberals” through a list of fairly irrelevant facts like decline in church attendance etc. Everyone in the room was fairly confused about just what the intention of the piece was.

Boucher and Sharpe’s argument is confused, but the intention is clear: to find any fact or argument that can be used to discredit ‘neoliberalism’ or ‘postmodern’ conservatism.

The point of mentioning declining church attendance, along with declining political party membership, lower levels of institutional trust, and rising divorce is to argue that there has been a decline in social capital, which Boucher and Sharpe hope to pin on ‘neoliberalism’.

In their discussion of social capital, they draw on Robert Putnam, and his book Bowling Alone. On p.169 our authors tell us that:

For Putnam, this [decline in social capital] cannot be solely attributed to the rise of neoliberalism since since 1973. [italics added]

Actually, Putnam thinks that hardly any of social capital’s decline is due to market economics. He dismisses its role in two pages of Bowling Alone (pp.282-83), conceding only a loss of civic leadership as small town businesses are replaced with giant corporations. His main objection is that America has been a market society for centuries, during which social capital has gone up and down. ‘A constant can’t explain a variable’, he says.
Read the rest of this entry »

How bad is my quality of life?

According to this Bankwest quality of life ranking, the Melbourne local government area, which includes Carlton, has the lowest quality of life in Victoria, and one of the lowest rankings in the whole country.

But I don’ t want to live anywhere else. Am I mad, or is this research bad?

To be sure, city living is not perfect. It can be a bit noisy. In Carlton, the presence of public housing, charities, and hospitals serving the mentally ill means that observance of the social niceties is not as high as it might be in Ku-ring-gai, the top ranked local government area in the country. And of course deeply unsound political views prevail (though it is not as bad as the city of Yarra across the road).

But inner Melbourne has a huge amount going for it too. The mix of cafes, bars and restuarants is the best in the country. There is a an excellent selection of shops. I’m not a sports fan, but for those who are there is an unmatched concentration of sporting venues around the CBD. There are beautiful 19th century gardens (including one just down the street from me). There are plenty of cinemas and theatres. There are two good universities in or near the CBD.
Read the rest of this entry »

Does staff turnover cost 10% of GDP?

According to reports in today’s Fairfax papers

STAFF turnover is costing Australian businesses $100 billion a year in lost productivity, training and recruitment costs – and generation Y is copping much of the blame. …

A new study shows staff turnover for workers in their 20s is running at 40 per cent a year. The rate for all workers is 18 per cent.

$100 billion is about 10% of GDP, and more than double all of education’s contribution to GDP. It doesn’t seem very likely that staff turnover has such a negative effect on the economy, particularly as there is also a balancing positive effect in achieving a better match between jobs and workers.

And if 18% of all workers are replaced each year at a cost of $100 billion it means that every job change costs about $52,000. Maybe some high-level job turnover has costs at that level, but in the industries where staff churn is greatest, accommodation, restuarants, cafes and retail, I’m sure it is a very small fraction of that, given the relatively low levels of skill typically required and the fact that employees will bring skills from previous jobs to new jobs.

Gen Y is being unfairly blamed too. Read the rest of this entry »

A connies con

In another case of opinion masquerading as journalism, The Sunday Age yesterday led with a story headlined ‘It’s time to bring back the connies’. By ‘connies’ they mean tram conductors.

This is a story that periodically crops up as slow-news day nostalgia in the Melbourne media, but this one uses ‘research’ commissioned by the newspaper to bolster its case. The net cost of $12 million a year is based on some very optimistic assumptions about higher fare revenue, through reduced evasion and increased patronage.

It’s true that conductors would encourage dishonest tram users to buy a ticket. But conductors would reduce revenue from honest tram users who want to buy a ticket but cannot. The main problem with the tram system is peak-hour overcrowding. You simply could not get to the conductor most of the time to buy a ticket – particularly in the very long newer trams with narrow passageways above the wheels, which would make it difficult for conductors to walk up and down. It’s hard enough for passengers to get to the validating machines sometimes, but at least there are lots of them and they are near the doors.

Just like there are people who still prefer to line up between 9.30am and 3.30pm Monday to Friday to see a bank teller rather than use an at ATM at their convenience, there are people who prefer transactions with conductors to a brief encounter with a validating machine. Mostly lonely elderly people with too much time on their hands, I’d suggest. For the rest of us, conductors are just a nuisance, interrupting conversations and making you fumble around for your wallet after you have already sat down. And if trams are already overcrowded, how does an extra person on board help?

If there is a spare $12 million a year, spending that money on some extra capacity in the system would be a much better investment than an expensive exercise in nostalgia.

Prospect’s dubious list of top public intellectuals

The trouble with letting survey respondents select themselves is that the results can be very odd. How likely is it, for example, that even though the Muslim world does not have a single university in the world’s top 500, it nevertheless produces all ten of the world’s top ten public intellectuals, according to the latest Prospect public intellectuals poll?

I must confess to not even having heard of seven of the ten. And what has Tariq Ramadan done in the last three years to push him up from 58 to 8?

If we ignore the campaign to get Muslims to vote and delete the top ten, we have the same situation as in 2005 with Noam Chomsky, who apart from his loyal band of leftist followers is not taken seriously outside linguistics, as number one. Al Gore is number two, perhaps reflecting the fashionability of his issue.

Another problem is that the starting point is Prospect‘s list, with half of the top 10 and 17 of the top 50 in 2008 seemingly not even worth considering in 2005.

There is no easy way to conduct polls like this, but perhaps voters having to write in names without a predetermined list would both include intellectuals Prospect missed, and minimise blog-driven campaigns for particular individuals.

Though this is a marketing gimmick for Prospect, even gimmicks need a certain level of credibility. A list of top universities that put, say, Cairo University above Princeton is not going to be taken seriously. Whatever the merits of Fethullah Gülen, he is not the world’s top public intellectual, and Prospect lacks an even semi-plausible public intellectual ranking.

Same story, different headline

Sydney slips in city rankings

The Age, 11 June 2008

Sydney nicer than the other lot

, 11 June 2008

Believe it or not, companies pay for this rankings rubbish.

Update: Another day, another city ranking, with different results. And this one you can have for the price of a lifestyle magazine.

What we did not agree to at 2020

As Joshua Gans notes, the final report of the 2020 Summit is out.

The authors of the productivity stream report certainly have an interesting definition of the word ‘agree’, as in ‘the stream agreed to the following’. What this means is that the ideas that follow were not, in the limited time available, subject to sufficiently vigorous dissent to knock them out of consideration. But given those time constraints, most people were more concerned with getting their own pet ideas in than keeping other people’s pet ideas out.

Given the lack of a proper decision making procedure, the productivity stream as a whole agreed to none of the ideas presented.

There are many bad ideas in this document, but two in particular amused me:

* [ensure] that policies and programs are informed by evidence and rigorous evaluation

This from a group that rarely gave policy suggestions more than a few minutes of explanation or discussion.

* develop measures to improve work-life balance

From a group sacrificing its weekend, led by a man who shows less regard for work-life balance than just about any other major employer.

Could plastic bag use rise 40% in one year?

Media reports this morning are claiming that a ‘confidential draft report’ to the federal government shows that plastic shopping bag use soared last year:

Bag use dropped steadily to 3.36 billion a year in 2006, but spiked back up to 4.84 billion in 2007, the report said.

Now I can think of at least one reason that this report is ‘draft’: that number fails a basic ‘does it look right?’ test. Despite the active efforts of retailers to reduce plastic bag use – supermarkets with their canvas bags, other retailers switching back to paper – and consumers declining additional bags when a new item can go in an existing bag, per capita use actually increased 40% in just one year?

The people writing that report would need a very good theory to explain that before I would believe it. The most obvious explanation I can think of is that either or both of the 2006 and 2007 figures are wrong.

Such a shonky set of numbers is worthy of re-opening my old Catallaxy-era ‘dubious research’ category.