On Friday a number of UK publications posted news articles about a Deutsche Bank report saying that London’s public transport fares are the highest in the world.
- Evening Standard: London travelcards ‘the most expensive in the world by a large margin,’ study finds
- BBC: London’s monthly travel cost ‘most expensive in world’
- London Loves Business: Deutsche Bank study reveals London travelcards are the most expensive in the world
- The Times: London travelcard is most expensive in world (and forget driving)
- The Sun: RAIL-LY PRICEY London travelcards are the most expensive in the world… twice the price of a New York pass
- Yahoo: London plunges down quality of life index as cost of travelcards, renting and jeans hits hard
The comparison was of a monthly public transport pass.
Most of the news reports listed only the top three most expensive cities; London, Dublin and Auckland, and the cheapest: Mumbai.
I wondered where Melbourne ranked. The BBC report was the only one to list beyond the top three; it showed Melbourne at number 9, with a monthly Pass costing £82 (US $106).
It was also the only one to link to the source Deutsche Bank report. The report is not about public transport specifically, it’s about cost of living in numerous categories, in 47 cities. (Note: 47 cities is not the entire world.)
Looking through the Deutsche Bank report, they’d got figures from various places. The public transport fares had come from a web site called Expatistan, which compares cost of living indicators — indeed, Deutsche Bank credits them for most of their pricing data:
Most of our price data is collected from Expatistan (www.expatistan.com). We would like to give special thanks to the founder Gerardo Robledillo for permitting us to use this data and for assisting us in collating historical data from the website.
Clicking through to have a look at their page about Melbourne, I found they had a monthly public transport pass listed at A$136.
The problem is, this is wrong. A 30-day Pass actually costs $147.60 (excluding the cost of the Myki ticket). It looks like their price is about two years out of date.
Edit: It’s been pointed out that a 28-day Pass is about the right price. But 28 days isn’t a month (usually!)
Expatistan’s information is comprehensive… but it’s also crowd-sourced. This can be problematic — Deutsche Bank notes:
Much of our data is from sources that utilize crowdsourcing techniques to collect and aggregate price data. While this methodology provides them with regular data updates from a large sample, there may be issues with data quality and consistency over time. … Do note thus that prices, changes and ranks should be considered representative with considerable room for measurement and sampling error.
A few clicks and I was able to update the Melbourne price on Expatistan, but it’s not being accepted until I guess others enter the same data.
Sydney was ranked 7th most expensive by Deutsche Bank. Expatistan says it’s A$147, but it’s not clear how this was calculated given there is no monthly pass available. The weekly cap on Opal is A$60, so a monthly cap would be at least $240 for four weeks.
Even the provenance of the London figure quoted in all those articles is uncertain. The Deutsche Bank figure quoted is US$174. The Expatistan figure is UKP £134, which matches the USD figure. But £134 doesn’t match anything on the current official price list – the closest logical match is a zone 1 Monthly ticket, which is £126.80.
Zone 1 only covers central London. The New York City price doesn’t match the official price (USD $121) either, perhaps because it changed in March. But even if it did, that’s not really a valid comparison, as the NYC fare covers the entire subway and local bus network.
So really the fare comparison part of the study is problematic.
Perhaps I’m over-thinking it, but when I did fare comparisons in the past for the PTUA, I went to official sources of fare information, and I was as careful as possible to ensure I was comparing apples with apples.
In this case, the media quotes Deutsche Bank, who used Expatistan, who trust in their users and their algorithms. And data points I’ve checked are wrong, and/or, in my opinion, not equivalent enough to be compared.
The lesson here is: Don’t believe everything you read.
And when a study boldly states a conclusion, do a little digging, to understand how they got there.
6 replies on “The perils of comparison studies”
And most importantly, don’t trust apparently ironclad figures from big-name sources based purely on the organisational reputation of the source in other fields. This kind of ‘source laundering’ (Big name A cites smaller name B cites Joe Random C) is more common than you might think, and occurs particularly when a ‘gun for hire’ consultant or quango steps outside their primary area of expertise without actually bringing in other experts as associates or subcontractors.
(‘You’ meaning people in general BTW – not actually directed at you Daniel!)
I’m pretty sure Wellington is the most expensive place to use public transport in the world. There, a 2-3km trip costs you $5. The monthly pass for the inner zone (equivalent in area to Zone One here) is $150 in the local currency, steps up to $230 and $300, and then goes all the way up to $400 per month (AU$374).
Re: Sydney / Opal: @ $60pw the monthly cost is ($60*52w)/12m = $260 p.m.
Disclosure: I rented for many years, many years ago.
@George, the Wellington figure given is US$101.2. I don’t know what that correlates to, but the study claims it’s been dropping over the past few years. I can’t even see an all-modes Pass on the official web site.
[…] agree on both counts! But as usual it can be useful to trace back the source of the […]