Last week, the ACCC released its first monthly report looking at the state of broadband in Australia, with a particular focus on whether the NBN is delivering on the promises made by RSPs, as well as how ISPs delivering pre-NBN services are faring. The data, collected with SamKnows under the "Measuring Broadband Australia" project paints a positive picture although there is room for improvement.
Although the government has committed $6.6M over three years to test 4000 homes, just 600 volunteers have joined the testing program so far. That's an important thing to note given the NBN (National Broadband Network) rollout has reached almost 6.4 million premises with just under 3.7 million premised activating the service. (Source: NBNCo weekly report for week commencing 22 March 2018).
So, 600 sites from a potential pool of 3.7 million is only barely a representative sample. However, the ACCC (Australian Competition and Consumer Commission), which has about 9000 potential volunteers they are sorting through, is endeavouring for the pool of volunteers to be representative of the broad cross-section of internet users.
The battery of tests the SamKnows appliance is quite comprehensive and includes:
- Upload and download speed between SamKnows' test server and the computer running their agent software
- Latency and jitter
- Packet loss
- DNS testing
- A simulated load of a website that mimics real-world usage
By and large, the SamKnows data paints a reasonably positive picture. One of the big criticisms of ISP (Internet Service Providers) and RSP (Retail Service Providers) advertising has been the use of the nebulous term "up to" when describing the potential performance of a service. The ACCC's testing looks at how closely a service provider delivers on the promised performance and whether that changes markedly between peak and off-peak periods.
Given the limited pool of users in the test program, the ACCC's data focuses mainly on the four largest ISPs; iiNet, Optus, Telstra, and TPG. The data is broken out to separate NBN connections from ADSL.
All four NBN ISPs were delivering download speeds exceeding 80% of the maximum plan speed in both peak and off-peak periods. TPG, which has had a chequered relationship with the TIO (Telecommunications Industry Ombudsman) recorded the best performance, with average download speeds in excess of 90% of the maximum plan speed in both peak and off-peak periods.
Upload speeds were better, with iiNet, Optus and TPG all recording average upload speeds that were over 90% of the maximum plan seed. Telstra's results were around 86%.
Interestingly, NBN services showed very little variation throughout the day, even during the evening peak periods. This suggests that RSPs who were skimping on CVC (Connectivity Virtual Circuit) during the early days of the NBN program have cleaned up their acts. In addition, the ACCC's advice for RSPs to ensure claims they make in advertising seem to be working.
With ADSL services, the average download speed was around 8Mbps with uploads not quite reaching 1Mbps during normal and busy hours.
The ACCC suggests that packet loss that is under 1% is unlikely to affect users' browsing experience. According to their data, excessive packet loss occurred in 4% of the 218,000 tests conducted over the reporting period with no packet loss recorded in 74% of the tests. When that's broken out to compare NBN and ADSL services, ADSL services saw 6.4% of tests exceed 1% packet loss, whilst NBN only saw 3.4% of tests exceed 1% packet loss.
What's all this mean?
The short answer is that, based on the data received through SamKnows and published by the ACCC, NBN services are getting very close to delivering what customers are paying for. When customers order a 100/40Mbps service, they are getting performance that is very close to that level, receiving 80% - 90% of the expected speed with very little packet loss. However, there's no data in the ACCC's report regarding latency or jitter.
Clearly, the ACCC's strong action, forcing RSPs to purchase enough CVC to service new connections, rather than being able to under-provision their networks is taking affect.
Further, guidelines published by the ACCC seem to have had the desired effect with RSPs getting close to delivering on what they promise. By providing guidance on how to advertise NBN services, the ACCC has accomplished two things; they have "helped" RSPs advertise in a way to works for customers, and they have given themselves a handy tool to use should they have to prosecute an RSP for not delivering.
RSPs simply can't say "we didn't know what the ACCC wanted" when it's been laid out clearly for them.
However, we still don't have a massive pool of data to look at and there's no reporting on installation issues. For example, how many installations are completed on the first appointment with no loss of services? And how punctual are installers when arriving at a customer's site? Those issues have been significant in damaging the reputation of NBNCo.
NBNCo's reporting suggests that about 50,000 premises are added to the network each week, across the different connection options, with about 30,000 of new activations each week. How many of those are resolved with the first, on-time appointment? Based on reports made to the TIO, 27,000 people made complaints about the NBN last year with just under 4,000 of those related to speed. The rest related to service delays and unusable services.
This first report by the ACCC is largely good news. But there's more to the success of the NBN deployment than RSPs simply delivering the promised bandwidth.