The Hunter region around Newcastle, just north of Sydney, is the one place in Australia right now where both Optus and Telstra offer high-speed 4G mobile broadband connections. Which carrier offers the fastest speed? Road Worrier tested both providers (and the best available option from Vodafone) to find out.
The 4G Battle
Optus has been operating a trial of its 1800MHZ LTE services around the Hunter since April. Telstra's 4G LTE (also on 1800MHz) have been live since last September. Optus is planning to expand its service to capital cities later this year, at which point it will also announce pricing and availability its 4G offering. At the moment, however, the Hunter Valley is the only place the two services compete head-to-head (though to use Optus you have to be signed up as one of the 1,000 locals involved in its trial). As such, it seemed the ideal opportunity for Road Worrier to compare the two options head-to-head. The chief selling point of 4G is data speed, so that's the main area to measure.
Vodafone is also planning to launch its own 4G network, but not until 2014. However, the Hunter Valley was one of the first areas to see Vodafone's general network improvements, so the performance in that region should be relatively strong. The key word there is 'should': when Vodafone first announced network enhancements in the area last July, my own tests suggested it was still performing quite poorly. But that was almost a year ago, so there should have been scope for improvements.
As that experience demonstrates, Lifehacker (and Road Worrier) has a long history of testing broadband performance in the Hunter Valley area. That largely reflects the ease of travelling through the area by train. As well as general trips through the region, some armed only with a BlackBerry, I've tested high-speed 3G performance for Telstra around Maitland. Options have been getting better over the past couple of years, so how much difference will 4G actually make? And is Vodafone still in the race?
Our Testing Methodology
For this set of tests, I travelled to the Hunter on Saturday and assessed performance for Optus, Telstra and Vodafone in four locations:
- Newcastle, the centre of the largest population base in the region;
- Maitland, one of the eastern-most points in Optus' coverage area;
- Hamilton, also near central Newcastle (and a major rail interchange point);
- Wyong, which is about as far south as the service reaches (and also a busy commuter town).
For each provider, I performed three speed tests using Speedtest.net and averaged the results. Speedtest isn't a perfect service and if anything tends to overestimate performance, but it provides a consistent basis for comparison. Saturday isn't a peak day for mobile traffic, so these very much represent best-case results. The same computer was used to run all tests. Speedtest measures three figures: ping times (in ms, the lower the better), download speeds (in Mbps, the higher the better) and upload speeds (in Mbps, the higher the better).
For Optus' service, I used the supplied Sierra Wireless 4G dongle, which is essentially the same hardware as Telstra's equivalent 4G dongle. (Optus is also providing a router which the dongle can be plugged into for home use and sharing amongst multiple machines, but that was irrelevant to an on-road test since it requires a separate power supply.) For Telstra connections, I used the 4G Wi-Fi hotspot, which can fail over to 3G if there's no 4G coverage. For Vodafone, I used the Pocket WiFi 2 hotspot, which covers the full range of frequencies Vodafone runs, ensuring it should get the highest available speed as well.
I travelled between each of these locations by train (and was pleased to hear announcements for quiet carriages on relevant services), then performed the testing on platforms. All the stations are relatively centrally located, and providing services at stations for waiting commuters is an obvious area where companies will want their 4G options to perform well.
Unlike some previous excursions into the Hunter, I didn't specifically carry out a 'train torture test' where I assessed how well each connection worked while on the move, since I was comparing multiple services and there wouldn't be a fair basis for doing that. However, on one segment of the journey (between Hamilton and Wyong), I did leave the Optus 4G dongle connected to see how well it would handle variations in signal. That's relevant because during the trial phase, the 4G service offered by Optus doesn't fail over to 3G. If there's no 4G, you get nothing. That won't be the case for the commercial service, but a 4G-only option does give a clear indication of how broad (or patchy) the coverage will be.
(Note: We didn't include vividwireless in these tests because the service is now effectively irrelevant after Optus acquired it earlier this year, it never offered coverage in the Hunter area and its general performance has frequently been abysmal.)
Click the table for a larger view
You can see the results for the four locations in the table above. I haven't averaged the performance to a single figure for each carrier, since that would be meaningless. However, there are clear trends evident in the figures:
- Ping times: Ping times measure basic server responsiveness, and are particularly relevant for voice-based services and gaming. Optus actually has the edge here, but even the worst-case option isn't a terrible figure.
- Download speeds: When downloading large files or watching videos, this tends to be the relevant figure. Optus' figures are good; even in the slowest location, its speeds are better than standard ADSL (and at the low end for ADSL2+). However, Telstra outperformed it in every single case, and in some locations by a very substantial margin. Vodafone's best-case performance of 2.3Mbps (in Newcastle) would be acceptable for basic services, but it clearly isn't in the race.
- Upload speeds: If you're sending files (especially video), upload speeds make a big difference. Optus again put in a solid performance, but was still half the speed (or less) of Telstra in every single location.
Because it's only in trial phase, Optus is in theory the least flexible provider. However, every test location we used had 4G service from Telstra as well, so the comparison of the 4G services is exactly that. That's not entirely surprising: Telstra announced that it was enhancing its 4G coverage in the Hunter immediately after Optus announced its trial, a move that appears designed to persuade potential customers not to make the switch.
Based on our testing, Optus' coverage seems relatively dense within the Hunter (it claims more than 90 base stations in the area). On the train between Hamilton and Wyong, there were just two locations where the signal disappeared altogether (around Dora Creek and Wyee). More pleasingly, the Sierra software managed to automatically reconnect when 4G was available again, which is something it has never managed to do in the past. That makes it a much more practical option for use on the go, regardless of the level of coverage.
The performance in Hamilton and Newcastle is worth contrasting. The two locations are less than 4 kilometres apart, but you wouldn't know that in the figures: Telstra and Optus scored some of their best results, while Vodafone had its worst performance. This underscores a vital point about 4G (and indeed 3G): performance can vary even in nearby areas, which is one reason that wireless options aren't ever going to completely displace wired connections. It's simply too hard to offer any kind of reasonable guarantee.
Vodafone's results still disappoint. It isn't a 4G provider yet, so its figures were always going to be lower than its rivals. But its performance is still poor. In the centre of Newcastle, where I performed similar tests a year ago, have improved somewhat. But the complete absence of signal in Wyong -- a major population centre -- underscores that Vodafone's basic availability still remains highly variable (and in a location where both its competitors posted very high scores). It's hard to imagine convincing anyone in the Hunter (or the Central Coast) that paying for a Vodafone connection would make sense, especially when its 4G timetable is not yet well-defined.
What Have We Learned?
The speeds we saw reinforce the prevailing viewpoint: Telstra has the fastest network, followed by Optus, followed (a long way behind) by Vodafone. In coverage terms, there didn't seem anything to separate the two 4G carriers (Vodafone, as we have already mentioned, has some clear and ongoing problems in that area despite having the longest-established network.)
In terms of speeds generally and upload speeds in particular, Telstra is well ahead of Optus -- and that's with a network in active use, not one being trialled ahead of a broader rollout. Speeds on any 4G network will inevitably drop as more users sign up, but Telstra appears to have an advantage even before that happens.
Performance is only one part of the equation. You can't buy Optus' 4G service just yet, so it isn't clear how it will compare in value terms to Telstra. That said, I'd be genuinely surprised if Optus didn't adopt the same approach as Telstra: not charging extra for 4G provided you have a device that can connect to it. In that scenario, Optus' slightly lower data charges might seem a worthwhile trade-off for some customers. Telstra provides a high-speed connection, but it certainly makes you pay for it.
Lifehacker Australia editor Angus Kidman isn't bitter that 4G is faster than his home ADSL connection. OK, that's a lie. His Road Worrier column, looking at technology and organising tips for travellers, appears each week on Lifehacker.